Methodological Considerations in Generating Provider Performance Score

This paper is intended for use by Chartered Value Exchanges (CVEs), community collaboratives, and other organizations interested in creating public reports on the performance of health care providers in their communities. This paper was written in response to a dilemma identified at the July 2009 National Meeting of CVEs hosted by AHRQ: Two organizations could use the exact same data to produce divergent public performance reports that send conflicting messages to patients and providers. At the request of CVE stakeholders, AHRQ commissioned RAND Corporation to develop a white paper to identify the key methodological decision points that precede publication of a performance report and to delineate the options for each. Our overall aim in developing this white paper is to produce a useful resource for CVEs and other collaboratives as they consider the range of available methodological options.

While decisions about methods are important, this paper also emphasizes the important roles that other factors (e.g., the goals of community stakeholders) play in determining how performance reports can be created. Figure 1 presents a simplified illustration of where methodological decisions fit into the overall process of creating performance reports.

Guided by the results of negotiations between stakeholders and the available data, a CVE can begin to make decisions about methods for producing provider performance reports. These decisions, which are the focus of this paper, generally do not have "right answers" based on methodological criteria alone. Therefore, this paper is designed to help CVEs consider options at each methodological decision point and understand the advantages and disadvantages associated with these options. CVEs can reopen stakeholder negotiations and obtain new data if no methodological option produces a performance report that is satisfactory to all stakeholders.

This paper's lists of advantages and disadvantages of each option are unlikely to be exhaustive. Readers may think of new advantages and disadvantages for many of the options. Moreover, the relative importance of each advantage and disadvantage will probably differ among CVEs. Therefore, the lists of advantages and disadvantages in this paper should serve as starting points for discussion.

Return to Contents 

Types of Measures, Providers, and Data

The methodological considerations reviewed in this paper may apply to a wide variety of performance measures, including measures of quality, costs, patient experience, and health outcomes. These methodological considerations also may apply to reports that focus on different kinds of providers, including individual physicians and other practitioners, small practices, large provider groups, and hospitals. Some of the methodological considerations are most applicable to certain types of data (such as health plan claims) and less applicable to other types of data (such as patient surveys). Similarly, some methodological considerations matter more when the providers being measured serve relatively small patient populations (e.g., individual practitioners). When a particular methodological choice pertains mainly to one type of data or one type of provider, we identify these situations.

Return to Contents 

Definition of "Provider"

Throughout this document, the word "provider" is intended to be flexible in its meaning. "Providers" may refer to individual health care practitioners (physicians, nurses, therapists, pharmacists, etc.), practices or clinics (i.e., collections of practitioners who provide care together at a single address), or larger health care organizations (physician groups, hospitals, nursing homes, etc.). Throughout the paper, we mention particular types of providers as illustrative examples of larger methodological points. However, all of the issues discussed in this paper can apply to multiple types of providers.

Page last reviewed September 2011
Internet Citation: Introduction: Methodological Considerations in Generating Provider Performance Score. September 2011. Agency for Healthcare Research and Quality, Rockville, MD.