Inter-rater reliability of preventable death judgments

This study examined the inter-rater reliability of preventable death judgments for trauma. A total of 130 deaths were reviewed for potential preventability by multiple panels of nationally chosen experts. Deaths involving a central nervous system (CNS) injury were reviewed by three panels, each consisting of a trauma surgeon, a neurosurgeon, and an emergency physician. Deaths not involving the CNS were reviewed by three panels, each consisting of two trauma surgeons and an emergency physician. Cases for review were sampled from all hospital trauma deaths occurring in Maryland during 1986. Panels were given prehospital and hospital records, medical examiner reports, and autopsy reports, and asked to independently classify deaths as not preventable (NP), possibly preventable (POSS), probably preventable (PROB), or definitely preventable (DEF). Cases in which there was disagreement about preventability were discussed by the panel as a group (via conference call). Results indicated that overall reliability was low. All three panels reviewing non-CNS deaths agreed in only 36% of the cases (қ = 0.21). Agreement among panels reviewing CNS deaths was somewhat higher at 56% (қ = 0.40). Most of the disagreements, however, were in judging whether deaths were NP or POSS. Agreement was higher for early deaths and less severely injured patients. For non-CNS deaths agreement was also higher for younger patients. When both autopsy results and prehospital care reports were available reliability increased across panels. A variety of approaches have been used to elicit judgments of preventability. This study provides information to guide recommendations for future studies involving implicit judgments of preventable death.