Assessment Tools for Use During Anesthesia-Centric Pediatric Advanced Life Support Training and Evaluation

Scott C. Watkins, Paul J. Nietert, Elisabeth Hughes, Eric T. Stickles, Tracy E. Wester, Matthew D. McEvoy

Research output: Contribution to journalArticlepeer-review

4 Scopus citations


Background Pediatric perioperative cardiac arrests are rare events that require rapid, skilled and coordinated efforts to optimize outcomes. We developed an assessment tool for assessing clinician performance during perioperative critical events termed Anesthesia-centric Pediatric Advanced Life Support (A-PALS). Here, we describe the development and evaluation of the A-PALS scoring instrument. Methods A group of raters scored videos of a perioperative team managing simulated events representing a range of scenarios and competency. We assessed agreement with the reference standard grading, as well as interrater and intrarater reliability. Results Overall, raters agreed with the reference standard 86.2% of the time. Rater scores concerning scenarios that depicted highly competent performance correlated better with the reference standard than scores from scenarios that depicted low clinical competence (P < 0.0001). Agreement with the reference standard was significantly (P < 0.0001) associated with scenario type, item category, level of competency displayed in the scenario, correct versus incorrect actions and whether the action was performed versus not performed. Kappa values were significantly (P < 0.0001) higher for highly competent performances as compared to lesser competent performances (good: mean = 0.83 [standard deviation = 0.07] versus poor: mean = 0.61 [standard deviation = 0.14]). The intraclass correlation coefficient (interrater reliability) was 0.97 for the raters’ composite scores on correct actions and 0.98 for their composite scores on incorrect actions. Conclusions This study provides evidence for the validity of the A-PALS scoring instrument and demonstrates that the scoring instrument can provide reliable scores, although clinician performance affects reliability.

Original languageEnglish (US)
Pages (from-to)516-522
Number of pages7
JournalAmerican Journal of the Medical Sciences
Issue number6
StatePublished - Jun 1 2017
Externally publishedYes


  • Anesthesia
  • Assessment
  • Interdisciplinary Education
  • Simulation
  • Teamwork

ASJC Scopus subject areas

  • General Medicine


Dive into the research topics of 'Assessment Tools for Use During Anesthesia-Centric Pediatric Advanced Life Support Training and Evaluation'. Together they form a unique fingerprint.

Cite this