Evaluation team

Project Leads: Professor Jonathan Kay and Professor Abdul Roudsari.
Research Associates: Kate Goddard (on site at Mayday), Dr Omid Shabestari, Juan Adriano and Alireza Golchin.

Background

Automation of healthcare processes is an emergent theme in the drive to increase patient safety. The Mayday Healthcare NHS Trust in Croydon, South London, was chosen as the pilot site for the implementation of the ECTMS to track blood from the point of requesting to the final transfusion and to ensure correct  matching of patient and blood. The benefits to be gained by the deployment of an electronic blood tracking system include reducing inpatient errors and safety incidents, reducing the number of specimens rejected by the lab, automated information checking, enhanced accountability and audit trail, and improved traceability of blood products.

Aims of the evaluation

Areas being covered in this evaluation include:

• The system – hardware, software and related procedures and protocols.
• The actions required to design and install the system. 
• The project management, technical and administrative skills and support needed to implement the system.
• The changes in clinical and other working practices required to implement the system. 
•  The response of clinical and other staff to the system. 
• Patient perception and satisfaction.
• The efficiency, effectiveness and reliability of the system particularly in terms of ensuring the correct matching of patients and blood.
• Any deficiencies or problems identified with the system and its use.
• A cost effectiveness analysis of the system design, planning, implementation and operation.
• User ideas for capturing and sharing the learning from this experience.

The full blood safety tracking pilot system went live in November 2009. The evaluation project will be assessing the system for a full year from go-live. End deliverables from the evaluation include a full report, an overview and a summary.

Implementing BloodTrack using tablet PCs, mobile carts, Bluetooth and passive RFID technology constituted new developments for the blood tracking system supplier and involved a considerable amount of work. This contributed to the go-live date being put back from April 2009 to November 2009.

Furthermore, the requirement to incorporate a unique identifier component and integrate BloodTrack with LIMs amounted in a large body of work for the Trust LIMS provider. Deliverables were impacted upon by the need to complete unforeseen developments to comply with guidelines driven by the MHRA and DH, and the preference to minimize the number of software versions delivered to the Trust.

Methodology

The Project Research Objective Evaluation (PROBE) framework (produced for the NHS Information Authority) was used to guide the relevant research questions and objectives according to five evaluation areas: strategic, financial, human, operational and technical. This framework was merged with a classic model to assess information systems success, the DeLone and McLean framework (updated in 2003) , which was used to generate lower-level attributes. This looked at six interrelated dimensions of success: system, service and information quality, use/intention to use and user satisfaction, and finally net benefits (hierarchically from individual to overall system). A cost benefit analysis is also being undertaken.

Assessment of the adherence to NHS CFH, National Patient Safety Agency (known as NPSA) and Medicines and Healthcare products Regulatory Agency (MHRA) standards and guidance is also being carried out.

Interviews and surveys were carried out with the implementation team, end user staff and patients to assess perceptions and satisfaction levels.

Quantitative methods are being used to track the effects of the blood safety tracking system, mainly via the Mayday’s own pathology database system as a source of data. The rate of uptake of the new system is key to the concept of future transferability to other trusts and is being primarily assessed by examining changes in usage in the old versus new system over time. Numbers, rates and reasons for blood request, blood usage and blood samples rejected by the lab are also being tracked, as are adverse incidents reported by the Mayday transfusion team.

Current progress

The evaluation project gathered baseline information for different sections of the process for blood transfusion. Interviews and questionnaires were carried out with the implementation team at Mayday University Hospital to gauge stakeholder opinion. A questionnaire was circulated amongst staff and potentially patients, exploring the importance of system success factors in anticipation of the full system going live.

Patient views were gathered about awareness of the blood tracking process, satisfaction with their experience and pros and cons of barcode and radio frequency identification (RFID) technology. A wristband audit was carried out, auditing the existence and the viability of inpatient wristbands.

[1] Heathfield, H., Clamp, S., Felton, D.: PROBE. Project Review and Objective Evaluation for Electronic Patient Record Projects. For the NHS Information Authority, 2001

[2] DeLone, W., McLean, E.: The DeLone and McLean Model of Information Systems Success: A Ten-Year Update. J of Manag. Inf. Sys. 19:4, 9--30 (2003)