Assessing the Psychometric Validity of the Epistaxis Severity Score: Internal Consistency and Test–Retest Reliability

Background: The Epistaxis Severity Score (ESS) is the gold-standard patient-reported outcome measure for evaluating nosebleed severity in patients with hereditary hemorrhagic telangiectasia (HHT). To date, the ESS has been assessed only for content validity and concurrent validity. Objective: We evaluate the internal consistency and test–retest reliability of the ESS. Materials and Methods: After receiving institutional review board approval, we sent an online survey battery, including the ESS survey, to 305 (39% male) English-speaking HHT patients ≥18 years old at a single center. Of those, 140 (46%) patients completed the battery, and 110/140 (79%) reported epistaxis. Cronbach's alpha and correlation analyses were used to evaluate internal consistency. For the test–retest reliability evaluation, we recruited 69 HHT patients during HHT clinic to complete 2 self-administered ESS surveys 2 weeks apart. Participants also completed a modified Clinical Global Impression-Improvement scale with readministration of the ESS survey. We calculated the intraclass correlation coefficient in a 2-way mixed model with absolute agreement. Results: The ESS survey demonstrated low internal consistency (Cronbach's alpha = 0.495), suggesting that it measured multiple unrelated concepts. Factor analysis revealed 3 latent factors with moderate intercorrelation, suggesting the presence of 3 related but distinct constructs underlying the ESS. However, the ESS demonstrated excellent test–retest reliability (intraclass correlation coefficient = 0.955; 95% CI, 0.91-0.98). Conclusion: Although the ESS demonstrates high test–retest reliability, it may not adequately assess different dimensions of nosebleed severity. Additional correlated survey questions and sub-scores may be needed to increase internal consistency to accurately measure each component of epistaxis severity. It is necessary to acknowledge epistaxis severity from different dimensions and to consider evaluating individual ESS items separately for a comprehensive understanding.