Background on the Measures Development Process
National Healthcare Quality Report, 2002
In its reauthorization legislation, Congress mandated that the Agency for Healthcare Research and Quality (AHRQ) produce an annual report on health care quality in the United States (Public Law 106-129). The National Healthcare Quality Report (NHQR) includes a broad set of performance measures that can be used to monitor the Nation's progress toward improved health care quality.
The NHQR and the National Healthcare Disparities Report (NHDR) were designed as companion reports and use identical measures of health care quality to the extent possible with available data.
AHRQ followed a multistep process to identify a measure set for the first NHQR. The following briefly summarizes the original NHQR measures development process.
Establishment of a Conceptual Framework
AHRQ contracted with the Institute of Medicine (IOM) to work on a conceptual framework for the NHQR. The IOM formed a 14-member committee of leading experts in quality and quality measurement, chaired by Drs. William Roper and Arnold Epstein. The committee commissioned papers, held multiple meetings to discuss alternative conceptual models, and heard testimony from a wide variety of groups. Their recommendations for the NHQR were included in Envisioning the National Health Care Quality Report.
That report recommended a conceptual framework that can be depicted as a matrix including both dimensions of care (e.g., effectiveness, safety, timeliness, patient centeredness, equity) and patient needs (e.g., staying healthy, getting better, living with illness or disability, coping with the end of life). The conceptual framework is below.
Components of Health Care Quality
|Health care needs||Effectiveness||Safety||Timeliness||Patient centeredness|
|Living with illness or disability|
|End of life care|
Although equity was included in the IOM's conceptual framework, it is not explicitly reflected in the matrix above because it cuts across all cells. Where possible, the NHQR assessed breakdowns of measures of the components of health care quality by sociodemographic group, insurance status, and the like. Users should consult the companion National Healthcare Disparities Report for in-depth results of these issues.
Identification of Preliminary Measure Set
AHRQ formed an Interagency Workgroup of the Department of Health and Human Services (HHS) to identify candidate measures for the first NHQR. The group includes representatives from AHRQ, the Centers for Disease Control and Prevention and its National Center for Health Statistics, the Centers for Medicare & Medicaid Services, the Food and Drug Administration, the Health Resources and Services Administration, the National Institutes of Health, the Substance Abuse and Mental Health Services Administration, and the Office of the Assistant Secretary for Planning and Evaluation.
The Interagency Workgroup developed a "call for measures" that was sent to all relevant Federal agencies. The IOM issued a complementary call for measures to the private sector. About 600 measures were submitted for consideration in response to these calls.
The Interagency Workgroup identified a set of interim priority areas for the first NHQR based on selected priorities established in Healthy People 2010. These interim priority areas were used to begin fleshing out the conceptual framework. Select for the detailed conceptual framework used in this effort.
The NHQR Interagency Workgroup continues to provide advice on data and measures development for the report.
The Interagency Workgroup mapped the candidate measures into the conceptual framework. The measures within each category of care were evaluated for inclusion in two parts:
- Measures were selected to maintain consistency with existing consensus-based measure sets where possible.
- The Interagency Workgroup assessed candidate measures using criteria recommended by the IOM in the Envisioning report, which include: importance, scientific soundness, and feasibility. Based on these assessments the workgroup identified a preliminary set of about 140 measures for the first NHQR.
Although the IOM did not recommend that AHRQ explicitly examine the efficiency of care in the NHQR, the further development of efficiency measures remains under discussion for future reports.
Comments on Measures for the First NHQR
AHRQ requested input on the preliminary measure set from stakeholders and other interested parties through September 18, 2002. (A separate Federal Register notice was published in March 2003 requesting comments on proposed home health measures by April 23, 2003.)
The comment process included a public hearing sponsored by the National Committee for Vital and Health Statistics, a technical expert panel meeting on home health measures, and input from other key technical advisory groups. Of special interest to the Agency was input on the individual measures included and the overall balance, comprehensiveness, and robustness of the measure set.
While obtaining input, AHRQ began relevant data analysis for the measures in the preliminary measure set. Based on the input and the results of analyses, and with broad input from throughout HHS, final decisions were made on which measures would be included in the first NHQR.
Multiple Reporting Products
AHRQ conducted audience research during the NHQR development effort to clearly identify the distinct needs of potential users of the NHQR. Results showed that:
- Some users, such as policymakers, are interested in the "bottom line" and want the basic findings without a lot of details.
- Others, such as analysts, want more extensive information on methodology and findings. They especially want to access detailed tables of results that they can examine in depth.
The 2003 report to Congress was designed to meet the needs of both audiences. It included a short summary report that highlights the key points from the analysis (for the policy audience) and appendixes that include technical specifications and detailed tables for all of the measures (for the analyst audience). All appendixes, as well as full report text, are available on AHRQ's QualityTools Web site.