Are ChatGPT and large language models “the answer” to bringing us closer to systematic review automation?

Riaz Qureshi, Daniel Shaughnessy, Kayden A.R. Gill, Karen A. Robinson, Tianjing Li, Eitan Agai

Research output: Contribution to journalComment/debatepeer-review

Abstract

In this commentary, we discuss ChatGPT and our perspectives on its utility to systematic reviews (SRs) through the appropriateness and applicability of its responses to SR related prompts. The advancement of artificial intelligence (AI)-assisted technologies leave many wondering about the current capabilities, limitations, and opportunities for integration AI into scientific endeavors. Large language models (LLM)—such as ChatGPT, designed by OpenAI—have recently gained widespread attention with their ability to respond to various prompts in a natural-sounding way. Systematic reviews (SRs) utilize secondary data and often require many months and substantial financial resources to complete, making them attractive grounds for developing AI-assistive technologies. On February 6, 2023, PICO Portal developers hosted a webinar to explore ChatGPT’s responses to tasks related to SR methodology. Our experience from exploring the responses of ChatGPT suggest that while ChatGPT and LLMs show some promise for aiding in SR-related tasks, the technology is in its infancy and needs much development for such applications. Furthermore, we advise that great caution should be taken by non-content experts in using these tools due to much of the output appearing, at a high level, to be valid, while much is erroneous and in need of active vetting.

Original languageEnglish (US)
Article number72
JournalSystematic reviews
Volume12
Issue number1
DOIs
StatePublished - Dec 2023

Keywords

  • Artificial intelligence
  • Large language models
  • Methodology
  • Systematic review

ASJC Scopus subject areas

  • Medicine (miscellaneous)

Fingerprint

Dive into the research topics of 'Are ChatGPT and large language models “the answer” to bringing us closer to systematic review automation?'. Together they form a unique fingerprint.

Cite this