TY - JOUR
T1 - Developing an Assessment Framework for Essential Internal Medicine Subspecialty Topics
AU - Chida, Natasha
AU - Brown, Christopher
AU - Mathad, Jyoti
AU - Carpenter, Kelly
AU - Nelson, George
AU - Schechter, Marcos C.
AU - Rebolledo, Paulina A.
AU - Fabre, Valeria
AU - Cantillo, Diana Silva
AU - Longworth, Sarah
AU - Amorosa, Valerianna
AU - Petrauskis, Christian
AU - Boulanger, Catherine
AU - Cain, Natalie
AU - Gupta, Amita
AU - McKenzie-White, Jane
AU - Bollinger, Robert
AU - Melia, Michael
N1 - Funding Information:
Research supported by the Natural Science Foundation of the First Affiliated Hospital of the Xinjiang Medical University Hospital, Youth Fund Project (#2015ZRQN29) and the Xinjiang Urumqi Infection and Tumor Key Laboratory Open Topic (#WIT- 2013-01).
PY - 2018/6/1
Y1 - 2018/6/1
N2 - Background : Assessing residents by direct observation is the preferred assessment method for infrequently encountered subspecialty topics, but this is logistically challenging.Objective : We developed an assessment framework for internal medicine (IM) residents in subspecialty topics, using tuberculosis diagnosis for proof of concept.Methods : We used a 4-step process at 8 academic medical centers that entailed (1) creating a 10-item knowledge assessment tool; (2) pilot testing on a sample of 129 IM residents and infectious disease fellow volunteers to evaluate validity evidence; (3) implementing the final tool among 886 resident volunteers; and (4) assessing outcomes via retrospective chart review. Outcomes included tool score, item performance, and rates of obtaining recommended diagnostics.Results : Following tool development, 10 infectious disease experts provided content validity. Pilot testing showed higher mean scores for fellows compared with residents (7 [SD = 1.8] versus 3.8 [SD = 1.7], respectively, P < .001) and a satisfactory Kuder-Richardson Formula 20 (0.72). Implementation of the tool revealed a 14-minute (SD = 2.0) mean completion time, 61% (541 of 886) response rate, 4.4 (SD = 1.6) mean score, and ≤ 57% correct response rate for 9 of 10 items. On chart review (n = 343), the rate of obtaining each recommended test was ≤ 43% (113 of 261), except for chest x-rays (96%, 328 of 343).Conclusions : Our assessment framework revealed knowledge and practice gaps in tuberculosis diagnosis in IM residents. Adopting this approach may help ensure assessment is not limited to frequently encountered topics.
AB - Background : Assessing residents by direct observation is the preferred assessment method for infrequently encountered subspecialty topics, but this is logistically challenging.Objective : We developed an assessment framework for internal medicine (IM) residents in subspecialty topics, using tuberculosis diagnosis for proof of concept.Methods : We used a 4-step process at 8 academic medical centers that entailed (1) creating a 10-item knowledge assessment tool; (2) pilot testing on a sample of 129 IM residents and infectious disease fellow volunteers to evaluate validity evidence; (3) implementing the final tool among 886 resident volunteers; and (4) assessing outcomes via retrospective chart review. Outcomes included tool score, item performance, and rates of obtaining recommended diagnostics.Results : Following tool development, 10 infectious disease experts provided content validity. Pilot testing showed higher mean scores for fellows compared with residents (7 [SD = 1.8] versus 3.8 [SD = 1.7], respectively, P < .001) and a satisfactory Kuder-Richardson Formula 20 (0.72). Implementation of the tool revealed a 14-minute (SD = 2.0) mean completion time, 61% (541 of 886) response rate, 4.4 (SD = 1.6) mean score, and ≤ 57% correct response rate for 9 of 10 items. On chart review (n = 343), the rate of obtaining each recommended test was ≤ 43% (113 of 261), except for chest x-rays (96%, 328 of 343).Conclusions : Our assessment framework revealed knowledge and practice gaps in tuberculosis diagnosis in IM residents. Adopting this approach may help ensure assessment is not limited to frequently encountered topics.
UR - http://www.scopus.com/inward/record.url?scp=85050851712&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85050851712&partnerID=8YFLogxK
U2 - 10.4300/JGME-D-17-00377.1
DO - 10.4300/JGME-D-17-00377.1
M3 - Article
C2 - 29946392
AN - SCOPUS:85050851712
SN - 1949-8349
VL - 10
SP - 331
EP - 335
JO - Journal of graduate medical education
JF - Journal of graduate medical education
IS - 3
ER -