Professional designers informally use "war stories" - reports of events, problems, solutions and so on that occur during systems design and deployment projects - as a means of sharing knowledge and experience to enable one another to learn from past successes and failures. Given that sharing war stories has been shown to be important and useful for designers (Orr 1996) but is an informal and ad hoc activity our aim here is to assemble a resource of war stories relating to deployment for use by all stakeholders involved in the deployment process. By reading through the collection it is hoped that stakeholders can get an idea of the types of problems that may arise in a deployment process, when they may arise and how they were dealt with at the time. The principle behind this resource is the idea that to be forewarned is to be forearmed.
Lack of Code of Connections
Deploying With a Distributed Team is Difficult.
Off-the-peg did not fit like a glove.
Descriptions of Current Workflow and Datasets May be Unavailable and Difficult to Collect.
Configuration is the Most Problematic Design Phase.
Balancing the Requirements for Generic Integration of Processes with Local Variants is a Wicked Problem.
Different Suppliers can have Different Schedules of Delivery.
Relationships with Different Suppliers Change and may be Difficult to Control.
Who Are Your Users?
Changed Schedules May Lead to Difficult Relationships with Stakeholders.
Those You are Consulting May Not Realise That They are Being Consulted.
Suppliers May Have Their Eye on More Than Ball.
Training Well Before the Train Leaves the Station.
Birth is neither an Accident nor an Emergency
In Public Private Partnerships (PPPs) to design and deploy EPR systems the private supplier requires code of connections approval in order to enable access to the NHS network, through which the networks of individual hospitals can be gained. This technical security clearance is necessary for off-site access to the Trust’s networks and systems. In the case at Preston, code of connections approval was overlooked during initial project planning and preparation. With the private supplier being US based, and build going on simultaneously at two sites the lack of code of connections during the first six or so months of database build and configuration, work was hampered by the inability of paired US and UK analysts to share real-time up to date details of the system. Misunderstandings about the current configuration of the database delayed the project.
This problem clearly stemmed from a lack of initial understanding about what a PPP for designing an EPR would entail concerning access to the network infrastructure, and the requirements for off to on-site collaboration. Code of connections should be approved early on in the project.
Deploying With a Distributed Team is Difficult.
This problem was exacerbated by the code of connections problem but nevertheless it became clear that the working relationship between UK and US analysts could not be facilitated and sustained as successfully through remote modes of interaction (shared db access, telephone, teleconference, email, CMC) as originally conceived. Coordinating remote co-presence in offices was difficult due to transatlantic time differences, meaning that 9am in the US would be either 3 or 5pm in the UK, meaning synchronous collaboration could only occur for a few hours in the day. Lack of shared access to the current system also hindered this. Much collaboration was asynchronous, meaning emails or phone messages might only be picked up the next day and when US analysts attended a joint remote meeting (e.g. teleconference) at 8 or 9am their time they might not be up-to-date with the work having gone on (and any emergent problems) in the UK during that day. All of this made the collaboration more disjointed and hampered work. It should be noted that budgetary restrictions may work strongly against the possibility of having many of the providers’ workers on-site for large periods of the development (e.g. having to pay consultants away-from-home wages), however face-to-face, or at least same site collaboration is extremely important for development. The Preston solution (luckily achievable within contractual limits) was to fly the US analysts over for approximately one week a month for an intensive collaboration with the UK analysts. This was significantly more that originally planned fork.
Off-the-peg did not fit like a glove.
In the case of Preston the project was originally conceived as one where the US system could be relatively easily populated and configured to the requirements of the Trust. The US analysts would direct their UK equivalents to collect the required data sets and workflow information and then oversee the placement of these into an already organised hierarchical database. Just how off-the-peg a system for a large, complex, distributed organisation can be is an open question. Clearly this is not design from scratch but it is easy to underestimate the complexity of the task. See the following two hazards for further explanation.~There is clearly no simple solution to this hazard except to acknowledge its size and complexity and attempt to plan for this.
Descriptions of Current Workflow and Datasets May be Unavailable and Difficult to Collect.
Prior to the project there was no central (or in some cases even local) accurate description of the current workflows and datasets. Gaining access to the right members of staff to compile the correct datasets and workflow proved difficult for each module of the system. For historical reasons often these descriptions did not exist in any easily usable form, sometimes it was difficult gaining access to a dataset at all, or any person who was able to describe a workflow. In patient administration seemingly generic activities like managing clinics showed large variation making agreed upon descriptions hard to come by. Disputes and disagreements meant that during the build phase requirements, terminology and so forth were being continually revised.
This work should be undertaken prior to beginning an EPR project as it should make system selection and configuration easier. Although this was done in a very basic sense before tender, it became clear that there was either incomplete understanding or the information was incorrect.
Configuration is the Most Problematic Design Phase.
Although phases of design cannot be seen as completely discrete, database build and configuration has been the most problematic phase so far. As discussed above, although the system is to some extent off-the-peg, the work of describing current data sets and workflows, rationalising and transforming these and fitting them to the database model has been particularly difficult. This phase has run massively over-time taking at least twice as long as envisaged. This was partly due to the problems of adequately describing datasets and workflows in an agreed upon manner, partly due to requirements for integration of processes, and partly due to restrictions caused by the basic structure and configuration of the database (designed for the US insurance based system), terminology and so forth. Over time, new requirements emerge, are revised and work already completed places restrictions on further build or revision. Basically, accurately modelling the work of the Trust then transforming it and fitting it to the system, and adapting the database is complex activity involving compromises, satisficing, workarounds and winners and losers as some fixes work better for some groups of workers.
It would be tempting to place blame with the fact that the system was originally designed for the US, or that since the database is hierarchical it does not afford as much flexibility during configuration, however, part of the problem was to do with a lack of current understanding of datasets and workflows within the Trust as well as how those individual procedures would integrate. Since any generic solution will have to be tailored to any UK hospital, configuration is always likely to be the difficult phase. Better preparation concerning organisational knowledge of processes and how they integrate would make selecting a system and the job of configuration easier.
Balancing the Requirements for Generic Integration of Processes with Local Variants is a Wicked Problem.
Integration and the models of workflow that are instantiated in the database demand that certain processes are generic in character. For example clinics, or episodes or treatment should be initiated, terminated and flow in the same way, they need to be counted in the same way from reporting purposes, and adhere to the same model. Accommodating local practice in any implementation can be a hazard. The system has various assumptions and constraints built into it. Satisfying certain NHS requirements creates further constraints as to how local variants of practice may be expressed as interaction sequences for users. Furthermore, the agreed solution, although it may be a compromise for all users, will necessarily work better for some groups. As the build progresses, what has already been carried out also restricts whether emergent or revised local requirements can be accommodated, or how they can be accommodated. This all means that some local practices must change to fit the system, while others may only be accommodated by more complicated workarounds at the interface meaning users are faced with a more complicated local system in order to meet requirements for integration. Potentially in trying to serve local demands the solution is a compromise for all.
In the case of Preston the solution has been debate, discussion, compromise, satisficing and to a certain extent one would envisage this of every project. Again, prior preparation for integration could potentially have helped this problem.
Different Suppliers can have Different Schedules of Delivery.
EPR systems for the NHS will most likely have to be integrated with several legacy systems, or new systems supplied by different providers than the EPR. In the case of Preston, the main different systems were the various laboratory applications for analysing blood, tissue and cells and so forth. Initially, all these systems were meant to integrate with the new EPR using the new secure data exchange protocol HL7. This meant that these systems had to be upgraded. Since the EPR system was bought out by the supplier of some of the lab systems this was unproblematic for some of the systems. However, two suppliers of the systems originally suggested they would have an HL7 compliant version for September 2003, but failed to deliver or quickly communicate that they would be unable to do so. These integrations had to be removed from phase 1 deployment and workaround compromises created to make the systems usable in the meantime. In a more general sense problems like this can occur with software, hardware, data migration and so on, as different suppliers are contracted for different jobs. Ensuring delivery at the same time is difficult.
The solution at Preston was to institute contingency plans to develop workarounds, find other suppliers, or do extra work in-house to deal with late delivery from outside suppliers. Other solutions may be to produce tighter contracts with the suppliers.
Relationships with Different Suppliers Change and may be Difficult to Control.
With EPR systems being delivered via Public Private Partnerships (PPPs), the EPR deliverer is now responsible for managing the relationship with the suppliers of legacy systems. In the case of Preston this means that it is up to the provider to ensure that suppliers of legacy systems do the required work on their systems for the purposes of integration. This caused problems for the Trust as legacy providers adjusted to this new relationship. Particularly, now it was difficult to enforce that these third party suppliers were delivering on time.
The Trust solved this problem by putting pressure on the EPR provider to ensure that legacy providers were carrying out the required work for integration.
While it might seem straightforward to anticipate who your users might be, this is not always the case in practice. In one medical setting it quickly became apparent that student nurses would need to have access to a medical records system in order to fulfil the conditions of their course.
There were difficulties in making this access available because of the training requirements of the Trust, and the resource implications of managing a large number of short-term user accounts.
Do not just consider users in your organisation, but think also of organisations with whom you are affiliated who might require access to your system and consider special arrangements might need to be put into place in order to meet these requirements.
Changed Schedules May Lead to Difficult Relationships with Stakeholders.
Users and representatives of users who are involved in the project development, must manage their availability around their primary work responsibilities so it is important to ensure that when they are required to work on project duties their time has been properly negotiated and planned in advance, while trying to avoid changes to this. In Preston problems occurred as slippage meant that planned work and collaboration had to be cancelled and rescheduled a number of times. This caused tension between the project team and these staff members and their managers and logistical problems for re-scheduling.
The solution for the Trust was through careful negotiation and re-scheduling. More generally attempts should be made to make sure the schedule is set before arranging for stakeholders’ time.
Those You are Consulting May Not Realise That They are Being Consulted.
In the role-out of first a integrated care process and later a computer system to support that process, the developers thought that they were consulting widely, but during implementation staff on the ground complained that they had not been involved in any consultation. Staff evidently did not realise what the 'consulations' were about, what was at stake and felt that they had missed an opportunity to influence the final systems.
Recipient design consultations to the requirements of staff to ensure that they understand the scope of planned changes and what may be at stake.Build consultation into part of a wider requirements process.
Suppliers May Have Their Eye on More Than Ball.
Changing market conditions as well as supplier re-organisation can lead to a supplier's focus shifting from the product set that you use to other products or services. This can sometimes lead to work requested to ensure compatability with newer systems being delayed or remaining unfulfilled. Often (especially in the case of heavily used legacy systems) it is not easy to switch vendors or packages. Sometimes suppliers may be willing to face financial penalties if there is a big enough return from shifting resources elsewhere.
Where you are locked into a supplier for the maintenance and/or management of an existing system it is useful to attend user group meetings to maintain a picture of what is going on in the supplier organisation, particularly where their priorities lie. Where a difficulty arises it may be useful to align yourself with other customers with the same predicament to improve your leverage with the supplier.
Training Well Before the Train Leaves the Station.
When rolling out a large system then it can be difficult to coordinate training with deployment, as the latter may be delayed by unforseen circumstances. So when the system is rolled out, users may have forgotten the training they have received. They may need to repeat the training, or to go on a refresher course. A related hazard is:
Hazard(b) - Training is rarely comprehensive. Although training may give a grounding in the use of a system, deployment often (especially for medical applications) turns up cases where it is difficult to see how they should be properly recorded on the system. User's find this frustrating as they are typically keen to ensure that the work they have done is adequately recorded and represented. These problems can arise out of precise meanings of system terminology (for example, what is meant by a patient 'review').
Allocate resources for a local team member to become a 'local expert' on using the system. This person could both provide tutorial support in-situ, provide clarification on points of interpretation, and be a liaison point for the system developers.
Birth is neither an Accident nor an Emergency
(This story was told to me by Tom Lincoln.)
A very pregnant woman was admitted to A&E following an accident. Probably induced by the trauma, she gave birth in the A&E ward. This gave the computer the following problems (i) births could be registered only in the maternity ward. No other wards were given access to this facility. (ii) only patients admitted to A&E could be discharged or transferred from A&E; this therefore excluded the baby. The obvious solution, to arrange a virtual transfer for the mum to maternity from A&E so that she could give a virtual birth in maternity followed by a virtual transfer of mum and baby from maternity back to A&E was denied by the computer on the grounds that there were no spare beds for new admissions to maternity that night (true, as it happens). (Fortunately, a bed became available in maternity the next day, after mum was well enough to be moved out of A&E). 0The bed manager in maternity had to increase the number of available beds by one so that the virtual transfer for the purposes of giving birth could occur, then decrease the number of beds by one. Of course, both of these were reported by the computer to the director of clinical resources the next day.