TY - JOUR
T1 - Gastrointestinal endoscopy competency assessment tool
T2 - Reliability and validity evidence
AU - Walsh, Catharine M.
AU - Ling, Simon C.
AU - Khanna, Nitin
AU - Grover, Samir C.
AU - Yu, Jeffrey J.
AU - Cooper, Mary Anne
AU - Yong, Elaine
AU - Nguyen, Geoffrey C.
AU - May, Gary
AU - Walters, Thomas D.
AU - Reznick, Richard
AU - Rabeneck, Linda
AU - Carnahan, Heather
N1 - Funding Information:
DISCLOSURE: All authors disclosed no financial relationships relevant to this article. This project was supported by an American Society for Gastrointestinal Endoscopy Quality in Endoscopic Research Award. Dr Walsh is a doctoral fellow of the CIHR Canadian Child Health Clinician Scientist Training Program and the recipient of a Department of Paediatrics Research Fellowship (Hospital for Sick Children) and a Postgraduate Medical Education Award, University of Toronto.
Publisher Copyright:
© 2015 American Society for Gastrointestinal Endoscopy.
PY - 2015/6/1
Y1 - 2015/6/1
N2 - Background Rigorously developed and validated direct observational assessment tools are required to support competency-based colonoscopy training to facilitate skill acquisition, optimize learning, and ensure readiness for unsupervised practice. Objective To examine reliability and validity evidence of the Gastrointestinal Endoscopy Competency Assessment Tool (GiECAT) for colonoscopy for use within the clinical setting. Design Prospective, observational, multicenter validation study. Sixty-one endoscopists performing 116 colonoscopies were assessed using the GiECAT, which consists of a 7-item global rating scale (GRS) and 19-item checklist (CL). A second rater assessed procedures to determine interrater reliability by using intraclass correlation coefficients (ICCs). Endoscopists' first and second procedure scores were compared to determine test-retest reliability by using ICCs. Discriminative validity was examined by comparing novice, intermediate, and experienced endoscopists' scores. Concurrent validity was measured by correlating scores with colonoscopy experience, cecal and terminal ileal intubation rates, and physician global assessment. Setting A total of 116 colonoscopies performed by 33 novice (<50 previous procedures), 18 intermediate (50-500 previous procedures), and 10 experienced (>1000 previous procedures) endoscopists from 6 Canadian hospitals. Main Outcome Measurements Interrater and test-retest reliability, discriminative, and concurrent validity. Results Interrater reliability was high (total: ICC = 0.85; GRS: ICC = 0.85; CL: ICC = 0.81). Test-retest reliability was excellent (total: ICC = 0.91; GRS: ICC = 0.93; CL: ICC = 0.80). Significant differences in GiECAT scores among novice, intermediate, and experienced endoscopists were noted (P <.001). There was a significant positive correlation (P <.001) between scores and number of previous colonoscopies (total: ρ = 0.78, GRS: ρ = 0.80, CL: Spearman's ρ = 0.71); cecal intubation rate (total: ρ = 0.81, GRS: Spearman's ρ = 0.82, CL: Spearman's ρ = 0.75); ileal intubation rate (total: Spearman's ρ = 0.82, GRS: Spearman's ρ = 0.82, CL: Spearman's ρ = 0.77); and physician global assessment (total: Spearman's ρ = 0.90, GRS: Spearman's ρ = 0.94, CL: Spearman's ρ = 0.77). Limitations Nonblinded assessments. Conclusion This study provides evidence supporting the reliability and validity of the GiECAT for use in assessing the performance of live colonoscopies in the clinical setting.
AB - Background Rigorously developed and validated direct observational assessment tools are required to support competency-based colonoscopy training to facilitate skill acquisition, optimize learning, and ensure readiness for unsupervised practice. Objective To examine reliability and validity evidence of the Gastrointestinal Endoscopy Competency Assessment Tool (GiECAT) for colonoscopy for use within the clinical setting. Design Prospective, observational, multicenter validation study. Sixty-one endoscopists performing 116 colonoscopies were assessed using the GiECAT, which consists of a 7-item global rating scale (GRS) and 19-item checklist (CL). A second rater assessed procedures to determine interrater reliability by using intraclass correlation coefficients (ICCs). Endoscopists' first and second procedure scores were compared to determine test-retest reliability by using ICCs. Discriminative validity was examined by comparing novice, intermediate, and experienced endoscopists' scores. Concurrent validity was measured by correlating scores with colonoscopy experience, cecal and terminal ileal intubation rates, and physician global assessment. Setting A total of 116 colonoscopies performed by 33 novice (<50 previous procedures), 18 intermediate (50-500 previous procedures), and 10 experienced (>1000 previous procedures) endoscopists from 6 Canadian hospitals. Main Outcome Measurements Interrater and test-retest reliability, discriminative, and concurrent validity. Results Interrater reliability was high (total: ICC = 0.85; GRS: ICC = 0.85; CL: ICC = 0.81). Test-retest reliability was excellent (total: ICC = 0.91; GRS: ICC = 0.93; CL: ICC = 0.80). Significant differences in GiECAT scores among novice, intermediate, and experienced endoscopists were noted (P <.001). There was a significant positive correlation (P <.001) between scores and number of previous colonoscopies (total: ρ = 0.78, GRS: ρ = 0.80, CL: Spearman's ρ = 0.71); cecal intubation rate (total: ρ = 0.81, GRS: Spearman's ρ = 0.82, CL: Spearman's ρ = 0.75); ileal intubation rate (total: Spearman's ρ = 0.82, GRS: Spearman's ρ = 0.82, CL: Spearman's ρ = 0.77); and physician global assessment (total: Spearman's ρ = 0.90, GRS: Spearman's ρ = 0.94, CL: Spearman's ρ = 0.77). Limitations Nonblinded assessments. Conclusion This study provides evidence supporting the reliability and validity of the GiECAT for use in assessing the performance of live colonoscopies in the clinical setting.
UR - http://www.scopus.com/inward/record.url?scp=84929584808&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84929584808&partnerID=8YFLogxK
U2 - 10.1016/j.gie.2014.11.030
DO - 10.1016/j.gie.2014.11.030
M3 - Article
C2 - 25753836
AN - SCOPUS:84929584808
SN - 0016-5107
VL - 81
SP - 1417-1424.e2
JO - Gastrointestinal endoscopy
JF - Gastrointestinal endoscopy
IS - 6
ER -