Accuracy and Interrater Reliability of Point-of-Care Ultrasonography Image Interpretation for Intussusception

In Collaboration with the P2Network Intussusception Study Group

Research output: Contribution to journalReview articlepeer-review

Abstract

Objectives The aim of this study was to determine the accuracy and interrater reliability of (1) point-of-care ultrasound (POCUS) image interpretation for identification of intussusception and (2) reliability of secondary signs associated with intussusception among experts compared with novice POCUS reviewers. Methods We conducted a planned secondary analysis of a prospective, convenience sample of children aged 3 months to 6 years who were evaluated with POCUS for intussusception across 17 international pediatric emergency departments between October 2018 and December 2020. A random sample of 100 POCUS examinations was reviewed by novice and expert POCUS reviewers. The primary outcome was identification of the presence or absence of intussusception. Secondary outcomes included intussusception size and the presence of trapped free fluid or echogenic foci. Accuracy was summarized using sensitivity and specificity, which were estimated via generalized mixed effects logistic regression. Interrater reliability was summarized via Light's κ statistics with bootstrapped standard errors (SEs). Accuracy and reliability of expert and novice POCUS reviewers were compared. Results Eighteen expert and 16 novice POCUS reviewers completed the reviews. The average expert sensitivity was 94.5% (95% confidence interval [CI], 88.6-97.5), and the specificity was 94.3% (95% CI, 90.3-96.7), significantly higher than the average novice sensitivity of 84.7% (95% CI, 74.3-91.4) and specificity of 80.4% (95% CI, 72.4, 86.7). κ was significantly greater for expert (0.679, SE 0.039) compared with novice POCUS reviewers (0.424, SE 0.044; difference 0.256, SE 0.033). For our secondary outcome measure of intussusception size, κ was significantly greater for experts (0.661, SE 0.038) compared with novices (0.397, SE 0.041; difference 0.264, SE 0.029). Interrater reliability was weak for expert and minimal for novice reviewers regarding the detection of trapped free fluid and echogenic foci. Conclusions Expert POCUS reviewers demonstrate high accuracy and moderate interrater reliability when identifying intussusception via image interpretation and perform better than novice reviewers.

Original languageEnglish (US)
Pages (from-to)442-447
Number of pages6
JournalPediatric emergency care
Volume38
Issue number9
DOIs
StatePublished - Sep 1 2022

Keywords

  • intussusception
  • point-of-care ultrasound

ASJC Scopus subject areas

  • Pediatrics, Perinatology, and Child Health
  • Emergency Medicine

Fingerprint

Dive into the research topics of 'Accuracy and Interrater Reliability of Point-of-Care Ultrasonography Image Interpretation for Intussusception'. Together they form a unique fingerprint.

Cite this