Your browser doesn't support JavaScript. Please upgrade to a modern browser or enable JavaScript in your existing browser.
Skip Navigation U.S. Department of Health and Human Services
Agency for Healthcare Research Quality
Archive print banner

This information is for reference purposes only. It was current when produced and may now be outdated. Archive material is no longer maintained, and some links may not work. Persons with disabilities having difficulty accessing this information should contact us at: Let us know the nature of the problem, the Web address of what you want, and your contact information.

Please go to for current information.

Quality Interagency Coordination (QuIC) Task Force
Return to QuIC Home
About QuIC
Steering Group
Press Releases
Related Links
Site Map
Discussion Paper

Informing Employed Consumers About Health Care Quality (continued)

Task II. Improve the Supply and Delivery of Consumer-Oriented Quality Information

Purchasers and purchaser coalitions have played an integral role in the growth of publicly available information on the quality of health care. The focus of purchasers’ "report cards," "consumer guides," or "provider profiles" address the relative performance of four key facets of the health care delivery system:

  • Hospitals.
  • Physicians and other providers.
  • Hospital/physician organizations.
  • Health plans.

In the area of hospital performance, noteworthy efforts include the New York State Cardiac Surgery Reporting System, Pennsylvania Health Care Cost Containment Council Cardiac initiative, California Outcomes project, and the Cleveland Health Quality Choice project.9

Purchasers and coalitions have played a major role in performance reporting on both on the care system and health plan level. The Buyers Health Care Action Group (BHCAG) has disseminated extensive performance information on "care systems" in Minnesota. Health plan performance reporting efforts include those of major employers and coalitions including but not limited to:

  • General Motors.
  • DaimlerChrysler.
  • GTE.
  • The Pacific Business Group on Health.
  • Gateway Purchasers for Health.
  • The Greater Detroit Area Health Council.
  • Colorado Business Group on Health.

Purchaser reporting efforts often include data from national sources, such as the National Committee on Quality Assurance (NCQA) and other accrediting organizations. Some purchasers will supplement these data with information gathered through a Request for Information process conducted on their behalf by an outside entity such as a benefits consulting firm.

Purchasers have also provided consumers information to assist in the selection of a health plan at the annual open enrollment. Such information usually includes a comparison of the scope of covered services across the plan options, the amount of co-pays/deductibles, and differences in the workers’ contribution, as applicable. The purchasers also gave workers copies of the health plans’ provider directories, which were often larger than many telephone books. It was conventional wisdom that these types of information would aid workers in the selection of a plan.

In reaction to increasing health care costs, purchasers increased their efforts to encourage workers to select lower cost plans, such as health maintenance organizations (HMOs). Purchasers expanded open enrollment materials to include measures of "quality" on a health plan level. Measures of quality include:

  • Accreditation status by an accrediting body (e.g., NCQA, Amercian Accreditation HealthCare Commission/Utilization Review Accreditation Commission [URAC], Joint Commission on Accreditation of Healthcare Organizations [JCAHO], etc.).
  • Individual Health Plan Employer Data and Information Set (HEDIS) preventive services measure—immunization rates, mammography screening, etc.
  • Member satisfaction.

In the past 2 to 3 years, there has been a movement away from the reporting of individual HEDIS measures. The Foundation for Accountability (FACCT) undertook a number of studies to determine the type of health care information that consumers would find most useful. FACCT’s work found the following categories resonated with consumers:

  • Access and service.
  • Qualified providers.
  • Doctor communication.
  • Staying healthy.
  • Getting better.
  • Living with illness.

NCQA has embraced these categories for its consumer report card. Other major purchasers, such as General Motors, Ford Motor Company, and DaimlerChrysler have based their health plan quality reporting on largely the same FACCT categories.

Purchasers and others who wish to expand efforts to educate consumers about health care quality and to provide comparative performance information on quality face a number of barriers, as discussed below.

An Issue of Scope and Comparability

The majority of measures of health plan quality are currently limited to HMOs and point of service (POS) plans. Measures such as HEDIS, which are available and given audited data, are comparable across these types of plans. Unfortunately, HEDIS measures are not available for indemnity, preferred provider organization (PPO), or other types of health plans. Non-managed care plans, unlike managed care plans, have not been "required" by the marketplace to collect the types of HEDIS quality measures. Additionally, non-managed care plans often cite the methodological and cost concerns as barriers to collecting these data. Thus, purchasers are unable to provide consumers comparisons of plan performance across the various types of plans from which the consumer may choose.

A similar case also applies to measures of consumer satisfaction with a health plan. In the managed care environment, health plans’ use of the standardized CAHPS® (Consumer Assessment of Health Plans) tools yields comparable measures of consumer satisfaction across managed care plans. As with the HEDIS measures, comparable member satisfaction information is not available for non-managed care plans that do not utilize the CAHPS® tool.

Purchasers need to continue to work with the national accrediting bodies and others to encourage the rapid development of new measures. Given the cost of data gathering and methodological issues, consideration to community-wide data gathering efforts is warranted.

The Kitchen Table Problem

Historically, purchasers in a community have developed unique consumer reports of health plan performance. Since in a community, many households have family members who work for different companies, the potential for receiving different and often conflicting information is great. At open enrollment, dual worker families could have their respective employers’ health plan report cards side-by-side on the kitchen table, showing very different ratings for the same health plan.

For example, while three of the 1997 consumer report cards categories for Ford Motor Company and General Motors were similar, the scores were in agreement for less than 50 percent of the 40 health plans offered by both companies.10 Reasons for the variations in the scores between the two report cards include, but are not limited to, difference in the array of measures used to populate a category, assignment of different weights to measures within categories, and differences in the underlying distribution of the data since the aggregate set of plans are different.

In 1998, the three auto manufacturers, the United Auto Workers, the Greater Detroit Area Health Council (GDAHC) members, and the State of Michigan (both as an employer and as a purchaser of services for Medicaid enrollees) worked to develop a common consumer guide to alleviate the problem of conflicting information. Other major Michigan purchasers who are members of the local coalition, GDAHC, also adopted the common consumer guide. These purchasers include Detroit Edison, Meijers, the AFL-CIO and others. Using a common, community consumer guide can partially mitigate conflicting information on health plan performance.

Still, workers can be faced with conflicting health plan performance information even if communities embrace a common consumer report. Media, such U.S. News and World Reports, Consumer Reports, and others, develop ratings of health plans using unique methodologies. These reports often generate conflicting health plan performance results for the "same" performance category (e.g., staying healthy). Again, variations in the analytical frameworks used to generate these reports are the underlying cause of the variation in performance results.

Does One Size Fit All?

Until the recent advent of the Internet, purchasers relied on printed materials as the primary vehicle to disseminate health care information to consumers. Use of these printed consumer reports was built on a series of questionable assumptions. For example, when information was provided in a single format, it was assumed that the information and the format would appeal to the majority of employees/members. English proficiency, reading level, and understanding of health care were assumed to be largely homogeneous. Issues surrounding cultural diversity were not addressed.

Increasingly, purchasers are also using the Internet or their internal Intranet to provide information. Often, however, the purchasers’ use of the Internet/Intranet is still merely a replication of the print materials on a Web site. Some purchasers are starting to incorporate interactive consumer decisionmaking tools along with health plan information. These software tools enable workers to specify the relative importance of specific health plan attributes, including quality indicators. Using the worker’s desired weights, the software will rank the available plan alternatives in terms of meeting the employee/consumer’s needs.

Return to Contents

Task III. Develop Consumer-Oriented Measures

Reporting health care quality information to consumers is constrained by a lack of data. The issue of limited data is applicable to all levels of consumer decisionmaking: health plan (or care system), hospital, and physician. Further work is needed to develop and test performance indicators for all these areas that will be embraced by consumers.

The Health Plan Front

The development, testing, and implementation of new health plan measures are critical if purchasers are going to use value-based purchasing principles in the selection of a health plan(s) for their workers. The measures are also essential to empower consumers to make informed decisions regarding the selection of a health plan.

Data Gaps

Standardized measures that are applicable and available across all plan types are critical. Comparable measures of quality are not available across all types of health plans:

  • Indemnity.
  • PPO.
  • POS.
  • HMO.

While HEDIS and CAHPS® measures of plan performance are available for HMOs and POS plans, similar measures are not available for indemnity and PPO plans. Purchasers are unable to develop report cards that show performance difference across all plan types. Thus, consumers lack comparative performance information across the plan types.

But the Plans Are All the Same

Report cards of health plan quality often do not always show noticeable difference between health plan options. Frequently, plan performance within a geographic region is very similar, with the exception of a couple of outlier, low performing plans. This lack of differentiation between plans may be the result of:

  1. Highly aggregated reporting categories.
  2. Scarcity of data to populate the categories.
  3. Limited differentiation in the plan performance on individual measures.

The limited differentiation decreases the usefulness of the information to the consumer.

One Size Does Not Fit All

While significant progress has been made in testing the relevance of standardized measures with consumers, there is still much to learn. FACCT’s work has led to the development of a series of aggregate categories which testing found to resonate with consumers. However, research on what quality indicators are relevant to different sub-groups of consumers for different purposes is lacking. For example:

  • What quality measures do consumers with a chronic condition look for in choosing a plan?
  • Which consumers desire detailed data, and what data do they find useful?
  • What quality measures can be developed to assess home health, nursing home, and behavioral health?

The changing demographics of the workforce will also impact the manner in which quality data is reported. In 1995, "Generation Xers" (birth years 1961 to 1981) accounted for 79.4 million people. When compared to prior generations, Generation X is more diverse, ethnically, culturally, and economically. By the year 2020, projections from the U.S. Census Bureau estimate that non-Hispanic whites will comprise 65.2 percent of the total U.S. population.11 This is significantly down from the August 1999 estimate of 71.9 percent.12 The Generation Xers will likely differ significantly from the Boomers in their desire and use of quality measures.

Beyond the Health Plan

Consumers differ in the types of health care information that they want. When asked about the importance of a variety of health topics, consumers indicated the following ranking of health information in Figure 1 (1 KB).

Similarly, purchasers find that what their workers really want are quality measures for individual physicians and hospitals—not measures of health plan performance. While purchasers and coalitions have initiated projects to gather hospital- and provider-level data, concerns over the validity of the data have served as a barrier to their use in purchasing decisions. Lack of confidence in the data has also prevented wide dissemination and use by consumers. Yet the performance of hospitals and physician providers remains paramount in the eyes of many, and the development of appropriate quality measures is needed.

The recent focus by the LeapFrog Group on medical error prevention and patient safety is an excellent example of purchasers’ focus on hospital performance. Leapfrog purchasers and other purchasers and coalitions are requesting that health plans assess their networks compliance with the patient safety specifications. Hospitals are being asked to publicly warrant their compliance with the Leapfrog standards. Early experience with the Leapfrog standards has found that the data, as in the early stages of HEDIS, is difficult to collect and may suffer from a lack of comparability. Further refinement in the data specifications should quickly alleviate the data collection issues, with the ultimate goal of providing comparative performance information on patient safety to consumers.

Return to Contents

Task IV. Identify the Market Characteristics and the Purchasing Strategies that Support Consumer Use of Quality Information

A Choice of One

A key underlying premise of purchasers’ dissemination of quality information is to aid consumers in the choice of a health plan and/or health plan provider. The notion of choice is critical. Is there a role for purchaser dissemination of quality measures even if the employee/member does not have a choice of plan? The case for comparative health plan quality measures is difficult to make if the employee/member does not have a choice of plan. Absent a choice of plan, one can argue that consumers should have access to physician and hospital performance measures. These quality measures would not have to be developed by the purchaser. The measures could be developed on a community-wide basis by an independent organization. The purchaser could serve as one of multiple sources that communicate the availability of the information to consumers.

The Seesaw Effect

Purchasers providing workers with health care quality measures regarding plan and providers performance have not found compelling evidence that the information is used in the consumer’s decisionmaking process. This is especially true when there is little or no difference in the financial contributions required by the consumer.

This consumer behavior is consistent with advertising’s theory of low-involvement buying. Low-involvement buying is like a seesaw that is exactly parallel to the ground with two "equivalent" goods on either end. For example, consumer may view two brands of margarine as equivalent goods. Given that the consumer’s expenditures on these goods represents a very small portion of their disposable income, little time will be spent determining which brand to buy. Instead, the purchasing decision may be driven by the cost of the product, past purchasing habits, advertising, word of mouth, or other non-quality based factors.

A similar analogy can be applied to health care. From the purchaser’s perspective, the "purchase" of worker health care represents a significant expenditure. Thus, purchasers are "highly involved" in the purchase, carefully evaluating the advantages and disadvantages of the alternatives.

For the consumer, the choice of a health plan or provider is often a low-involvement purchase decision. On each end of the seesaw are two "brands" of health care plans or providers. In the eyes of the consumer, the two brands are very similar, if not identical. One can hypothesize that the consumer’ ultimate decision is made much as in the case of different brands of margarine—driven by price, past purchasing habits or advertising, word of mouth, or other factors unrelated to quality.

Clouding the Picture

Consumers’ and sometimes purchasers’ decisions regarding health plans can be made more difficult because of plan benefit design differences. Plan benefits in terms of covered services and out-of-pocket payments may influence consumer choice. Consumers may select a plan over an alternative because it has "richer" benefits–greater scope of covered services and/or lower required out-of-pocket expenditures. The selection of a plan is thus made absent differences in plan performance. Similarly, some health plans have different level of co-pays for preferred versus non-preferred providers. The distinction of preferred versus non-preferred typically reflect the plan’s ability to contract a lower reimbursement rate with the preferred provider. In selecting among providers, consumer’s choice may be based on cost differentials rather than quality differences.

Return to Contents

Task V. Evaluate the Impact and Utility of Consumer Information Efforts

Efforts to evaluate the impact of quality measures on employed consumers’ decisionmaking are extremely limited. Although purchasers provide much of the quality measure information to consumers with employment-based health care coverage, purchasers have done little to evaluate its impact. For the majority of purchasers, health care, while often-significant budget expenditure, is not their core business. Rather, health care is one of a number of "fringe" benefits provided so as to retain and attract workers. Thus, purchasers tend to rely on somewhat unsophisticated measures to assess the impact of the quality measures provided to workers. Such measures include changes in plan enrollment and level of worker satisfaction, without taking into account variables such as changes in worker health care contributions/co-pays, demographics, health plan offerings, etc.

Purchasers are reluctant to dedicate the necessary resources to formally evaluate the impact of quality measures on consumer decisions. Formal studies are expensive, time consuming, and may require staff expertise that the purchaser does not have in-house. It is extremely difficult for purchasers to make a business case that even if resources were dedicated to an evaluation process, a positive return on investment would be realized.

Consideration of a standardized framework for evaluation of consumers’ use of quality is needed. A standardized framework would offer the following benefits:

  • Provide purchasers and/or other organizations with an evaluation process that is easily implemented.
  • Allow comparability of purchaser’s or other organization’s evaluation results within and across markets.
  • Create an opportunity to identify purchasers' and other organizations' "best practices" in motivating consumers to use quality measures.
  • Reduce the costs of evaluation by establishing a common evaluation framework that could by used by all.

The Web site "Talking Quality " is a noteworthy step in making advice and resources available to purchasers and other organizations. Other opportunities that warrant consideration include the development/implementation of national consumer surveys as well as other forums for purchasers and other organizations to share the results of their evaluation efforts.

Return to Contents


1. Health United States 2000. Health care coverage of persons under age 65 years of age, according to type of coverage and selected characteristics: United States, selected years 1984-97.
2. Robert Wood Johnson Foundation Employer Health Survey, 1997.
3. VHA Report, Qualitative Consumer Research: Development of Quality Measurement and Reporting Program, Final Report, May 2000.
4. Epstein A. Public Release of Performance Data, A Progress Report from the Front. JAMA 2000;283:1884-6.
5. Leapfrog Web site:
6. Hibbard J, Slovic P, Jewett J. Informing Consumer Decisions in Health Care: Implications from Decision-Making Research. The Milbank Quarterly 1997;75:395-413.
7. VHA Report, Qualitative Consumer Research: Development of Quality Measurement and Reporting Program, Final Report, May 2000.
8. Hibbard J, Slovic P, Jewett J. Informing Consumer Decisions in Health Care: Implications from Decision-Making Research. The Milbank Quarterly 1997;75:395-413.
9. Marshall M, et. al. The Public Release of Performance Data—What Do We Expect to Gain? A Review of the Evidence. JAMA 2000;283:1866-74.
10. McGlynn E, et al. Creating a Coordinated Autos/UAW Reporting System (CARS), RAND Study Paper, 1999.
11. Source: Population Projections Program, Population Division, U.S. Census Bureau, Washington, DC. Projections of the Resident Population by Race, Hispanic Origin, and Nativity, Middle Series, 2016 to 2020. Internet Release Date: January 13, 2000.
12. Source: Population Estimates Program, Population Division, U.S. Census Bureau, and Washington, D.C. 20233. Resident Population Estimates of the United States by Sex, Race, and Hispanic Origin: April 1, 1990 to July 1, 1999, with Short-Term Projections to August 1, 2000 - Cont. Internet Release Date: September 27, 2000.

Return to Contents

Current as of October 2000

Return to Informing Consumers About Health Care Quality
QuIC Home Page
Department of Health and Human Services

The information on this page is archived and provided for reference purposes only.


AHRQ Advancing Excellence in Health Care