Clinical utility and functionality of an artificial intelligence-based app to predict mortality in COVID-19: Mixed methods analysis

Ahmed Abdulaal, Aatish Patel, Ahmed Al-Hindawi, Esmita Charani, Saleh A. Alqahtani, Gary W. Davies, Nabeela Mughal, Luke Stephen Prockter Moore

Research output: Contribution to journalArticlepeer-review


Background: The artificial neural network (ANN) is an increasingly important tool in the context of solving complex medical classification problems. However, one of the principal challenges in leveraging artificial intelligence technology in the health care setting has been the relative inability to translate models into clinician workflow. Objective: Here we demonstrate the development of a COVID-19 outcome prediction app that utilizes an ANN and assesses its usability in the clinical setting. Methods: Usability assessment was conducted using the app, followed by a semistructured end-user interview. Usability was specified by effectiveness, efficiency, and satisfaction measures. These data were reported with descriptive statistics. The end-user interview data were analyzed using the thematic framework method, which allowed for the development of themes from the interview narratives. In total, 31 National Health Service physicians at a West London teaching hospital, including foundation physicians, senior house officers, registrars, and consultants, were included in this study. Results: All participants were able to complete the assessment, with a mean time to complete separate patient vignettes of 59.35 (SD 10.35) seconds. The mean system usability scale score was 91.94 (SD 8.54), which corresponds to a qualitative rating of “excellent.” The clinicians found the app intuitive and easy to use, with the majority describing its predictions as a useful adjunct to their clinical practice. The main concern was related to the use of the app in isolation rather than in conjunction with other clinical parameters. However, most clinicians speculated that the app could positively reinforce or validate their clinical decision-making. Conclusions: Translating artificial intelligence technologies into the clinical setting remains an important but challenging task. We demonstrate the effectiveness, efficiency, and system usability of a web-based app designed to predict the outcomes of patients with COVID-19 from an ANN.

Original languageEnglish (US)
Article numbere27992
JournalJMIR Formative Research
Issue number7
StatePublished - Jul 2021


  • App
  • Artificial intelligence
  • COVID-19
  • Coronavirus
  • Development
  • Function
  • Graphical user interface
  • Machine learning
  • Model
  • Mortality
  • Neural network
  • Prediction
  • Usability
  • Utility

ASJC Scopus subject areas

  • Medicine (miscellaneous)
  • Health Informatics


Dive into the research topics of 'Clinical utility and functionality of an artificial intelligence-based app to predict mortality in COVID-19: Mixed methods analysis'. Together they form a unique fingerprint.

Cite this