How This Paper Is Organized
Methodological Considerations in Generating Provider Performance Score
This paper begins with a discussion of performance misclassification, which is a fundamental, overarching methodological issue in any report of provider performance. Performance misclassification is defined and briefly discussed in the next section. Readers who are interested in more detailed information about the ways performance can be misclassified are encouraged to consult two appendixes to the report:
- Appendix 1: Systematic performance misclassification.
- Appendix 2: Performance misclassification due to chance.
The remainder of this paper follows a series of six general steps that a CVE or other collaborative is likely to encounter when creating a performance report:
- Negotiating stakeholder consensus on "value judgments" of performance reporting.
- Selecting the measures that will be used to evaluate providers.
- Identifying data sources and aggregating performance data.
- Checking data quality and completeness.
- Computing provider scores.
- Creating performance reports.
Figure 2 shows this series of steps and some of the methodological decision points that may occur in each step. For each decision point, we present options. For most decision points, we also present examples of how a sample of CVEs (or stakeholder organizations) has chosen among the available methodological options. We focus on the reasoning behind these choices so that readers can get a sense of whether a given option may be preferable in their communities.
CVEs vary in the length of their reporting experience. Some CVEs have already produced multiple public reports and others have only recently begun to gather performance data for the first time. Therefore, this paper is organized so that it can be read from front to back, or readers can go directly to the section that pertains to a current report-making step. The full list of methodological questions addressed in this paper is available in the Contents.
This paper concludes with a summary of methodological decisions made by a sample of stakeholders from nine CVEs. These nine CVEs may not be nationally representative. However, the choices and reasoning expressed by their stakeholders may be useful as a reference point for discussions about performance reporting methods.