Human Factors Evaluation of the Universal Anaesthesia Machine: Assessing Equipment with High-Fidelity Simulation Prior to Deployment in a Resource-Constrained Environment

Background: Anesthesia providers in low- and middle-income countries face many challenges, including poor availability of functioning equipment designed to meet their environmental, organizational, and resource constraints. These are serious global health disparities which threaten access to care and patient safety for those who receive surgical care. In this study, we conducted a simulation-based human factors analysis of the Universal Anaesthesia Machine (UAM®), a device designed to support anesthesia providers in austere medical settings. Our team anticipated the introduction of the UAM® to the two major referral hospitals in Freetown, Sierra Leone. A prior observational study had identified these two hospitals as having environmental conditions consistent with an austere environment: an unstable electrical grid, as well as limited access to compressed oxygen, biomedical support, and consumables. Although the Baltimore simulation environment cannot reproduce all of the challenges present in a resource-constrained environment such as Sierra Leone, the major impediments to standard anesthesia machine functionality and human factors-associated use can be reproduced with the use of high-fidelity simulation. Using anesthesia care providers who have limited UAM® familiarity, this study allowed for the examination of machine-user issues in a controlled environment in preparation for further field studies concerning equipment introduction, training and device deployment in Sierra Leone. The goals of this study were: 1. to assess the usability of the UAM® (machine-user interface, simulated patient use, symbology, etc.) across different provider user groups during simulation of use in scenarios depicting routine use in healthy patients, use in clinically challenging patients and use in environmentally-challenging scenarios in a controlled setting devoid of patient risk, and 2. To gather feedback on available UAM manuals and cognitive aides and UAM usability issues in order to guide development of curricula for training providers on use of the UAM® in the intended austere clinical environments. Methods: Residents, fellows, attending physician anesthesiologists, student nurse anesthetists, and nurse anesthetists participated in a variety of simulations involving the Universal Anaesthesia Machine® at the Johns Hopkins Medicine Simulation Center between September 2012 and July 2013. Data collected included participant demographics, performance during simulation scenarios captured with critical action checklists, workload ratings captured with the National Aeronautics and Space Administration Task Load Index (NASA TLX), and participant reactions to UAM® use captured through a post-session survey and semi-structured usability debriefing. The scenarios were: 1. normal use (machine check, induction, and maintenance of an uneventful case), 2. use in a challenging clinical condition (acute onset of bronchospasm) and 3.use in an adverse environmental event (power failure). Critical action checklists and workload ratings were analyzed by Analysis of Covariance (ANCOVA) to control for participant demographics. Usability debriefings were analyzed qualitatively. Results: Thirty-five anesthesia providers participated in the study. Overall participant ratings, observations of performance in simulation scenarios, and usability debriefings indicated a high level of usability for the UAM®. Mean participant ratings were high for ease of use (5.4 ± 0.96) and clarity of instruction (6.2 ± 0.87) on a 7-point scale in which higher ratings indicate more positive perceptions. After adjusting for clinical experience, workload ratings were significantly higher in the bronchospasm scenario than in the normal/routine use (P = 0.046; 95% CI, 0.33–34.7) or power failure scenarios (P = 0.012; 95% CI, 5.24–37.9). Thirty-two specific usability issues were identified and grouped into five themes: device design and labeling, machine use during simulation scenarios, user-anticipated errors or hazards, curriculum issues, and overall impressions of the UAM®. Conclusions: The UAM® design addresses many of the key challenges facing anesthesia providers in resource-constrained settings. The simulation-based human factors evaluation described here successfully identified opportunities for continued refinement of the initial device design as well as issues to be addressed in future curricula and cognitive aides.