Foreword: Continued National Dialogue on Methodological Decisions in Generating Provider Performance Scores

Methodological Considerations in Generating Provider Performance Scores

For the past 3 years, the Agency for Healthcare Research and Quality (AHRQ) has provided technical assistance to 24 multistakeholder community quality collaboratives, which we refer to as Chartered Value Exchanges (CVEs). These CVEs support an agenda of quality transparency via public reporting of physician and hospital performance. At a recent national meeting of CVEs, community leaders expressed concern that currently two organizations could use the exact same dataset to produce a public report, but the two reports could produce and release diverging provider performance scores. Score differences could result from the way one or more measurement and data collection decisions are made.

To set the stage for continued regional and national dialogue, AHRQ commissioned RAND Corporation's Mark Friedberg and Cheryl Damberg to isolate and examine the set of decisions that collaboratives and other report sponsors face in the steps leading to the release of a public report.

These decisions, 20 in all, are grouped in this white paper in the following six categories:

  1. Negotiating consensus on goals and value judgments of performance reporting.
  2. Selecting measures that will be used to evaluate provider performance.
  3. Identifying data sources and aggregating performance data.
  4. Checking data quality and completeness.
  5. Computing provider-level performance scores.
  6. Creating performance reports.

For each decision, optional decision paths are laid out and the relative pros and cons of each are examined. RAND developed this paper in partnership with a panel of representatives from nine community quality collaboratives. These individuals provided local perspectives and real-world vignettes to illustrate optional paths for each decision.

Our overall aim in commissioning this paper was to produce a useful resource for community collaboratives and regional and national policymakers as together we build a network of quality transparency that supports consumer, provider, and payer engagement in quality and, ultimately, quality improvement. I hope this white paper informs ongoing deliberations, and I welcome your feedback (

I thank Mark Friedberg, Cheryl Damberg, and their colleagues at RAND Corporation for their comprehensive and scholarly approach and for their timeliness in developing this important resource for report sponsors across the country. I also thank the nine CVE representatives who shared their perspectives and provided case examples used in the paper and the impressive list of experts who provided critical feedback on an earlier draft.

This white paper is the latest in a series of coordinated efforts by AHRQ to support and enhance ongoing local and national dialogue related to data, quality measurement, and reporting. Visit for the menu of related AHRQ resources.

Peggy McNamara
Senior Fellow
Agency for Healthcare Research and Quality
September 2011

Page last reviewed September 2011
Internet Citation: Foreword: Continued National Dialogue on Methodological Decisions in Generating Provider Performance Scores: Methodological Considerations in Generating Provider Performance Scores. September 2011. Agency for Healthcare Research and Quality, Rockville, MD.