Expanding the uses of AHRQ's prevention quality indicators: Validity from the clinician perspective

Background: The Agency for Healthcare Research and Quality's prevention quality indicators (PQIs) are used as a metric of area-level access to quality care. Recently, interest has expanded to using the measures at the level of payer or large physician groups, including public reporting or pay-for-performance programs. However, the validity of these expanded applications is unknown. RESEARCH DESIGN: We conducted a novel panel process to establish face validity of the 12 PQIs at 3 denominator levels: geographic area, payer, and large physician groups; and 3 uses: quality improvement, comparative reporting, and pay for performance. Sixty-four clinician panelists were split into Delphi and Nominal Groups. We aimed to capitalize on the reliability of the Delphi method and information sharing in the Nominal group method by applying these techniques simultaneously. We examined panelists' perceived usefulness of the indicators for specific uses using median scores and agreement within and between groups. Results: Panelists showed stronger support of the usefulness of chronic disease indicators at the payer and large physician group levels than for acute disease indicators. Panelists fully supported the usefulness of 2 indicators for comparative reporting (asthma, congestive heart failure) and no indicators for pay-for-performance applications. Panelists expressed serious concerns about the usefulness of all new applications of 3 indicators (angina, perforated appendix, dehydration). Panelists rated age, current comorbidities, earlier hospitalization, and socioeconomic status as the most important risk-adjustment factors. Conclusions: Clinicians supported some expanded uses of the PQIs, but generally expressed reservations. Attention to denominator definitions and risk adjustment are essential for expanded use.