Evaluating the Impact of Value-Based Purchasing: A Guide for Purchasers
A recent report by the Institute of Medicine (Crossing the Quality Chasm: A New Health System for the 21st Century) identified a "chasm" between the quality of care we have and the quality of care we should have. Employers could be a powerful force for closing this gap, since they pay for much of the health care in the United States. Past research has shown that employers and employer coalitions have at least some of the tools they need to serve as a force for quality, and a growing number of pioneers indeed are developing and implementing strategies they hope will improve the quality and value of the health care they purchase. But to date we have very little evidence on the impact of such efforts:
- What strategies are effective?
- Under what circumstances?
- In what markets?
This lack of evaluation presents problems at both the program and policy levels. At the program level, it means that employers and coalitions find it hard to determine whether their own initiatives, much less the initiatives they might choose to emulate, are in fact meeting their intended goals. Given rising health care costs, strategies that cannot prove their impact are likely to be abandoned. At the policy level, the lack of evaluation leaves unanswered the question of whether one can rely on the current market mechanism for improving quality.
The health services research community can play an important role in helping to evaluate past value-based purchasing efforts. A recent AHRQ Program Announcement, "Impact of Payment and Organization on Cost, Quality and Equity," seeks to encourage such research. In the meantime, however, employers and coalitions have indicated that they need some tools to help them do "real time" tracking and assessment of the impact of their own efforts, so they can tell quickly which to keep and which to drop, and how to fine-tune their strategies. Evaluating the Impact of Value-Based Purchasing Initiatives: A Guide for Purchasers is intended to meet this immediate purchaser need. We hope employers and coalition leaders will try it out, tell us how well it is working and how it could be improved, and use it to improve and enhance their value-based purchasing efforts.
—Irene Fraser, Ph.D., Director
Center for Organization and Delivery Studies
Agency for Healthcare Research and Quality
One of the unique aspects of health care in the United States is the manner in which it is financed. Unlike many other countries, the United States does not have a publicly administered universal insurance program. Instead, health care is financed through a multitude of public and private insurance programs administered by purchasers. In the United States, the primary group purchasers include Medicare and Medicaid for public health care programs, and employers and purchasing cooperatives for private health care programs. Purchasers are defined as follows:
"Purchasers" are public and private sector entities that subsidize, arrange and contract for—and in many cases bear the risk for—the cost of health care services received by a group of beneficiaries.
In the past decade, many public and private health care purchasers have become more active in the purchasing process. Rather than simply writing checks to health insurers or health care providers, they are attempting to measure, monitor, and improve the quality they are receiving for the health care dollar spent. Although there are numerous ways purchasers have approached these tasks, they have been collectively termed value-based purchasing (VBP).
The need and opportunity for VBP has probably never been greater than it is right now. After several years of moderate growth in health care costs, recent estimates indicate that costs grew by nearly 7 percent from 1999 to 2000, outpacing the growth of the economy for the first time in almost a decade (Levit, Smith et al., 2002). Reflecting that growth, private employers estimate that their health care costs in 2002 will be nearly 14 percent more than they were in 2001 (Ceniceros, 2001). These increases are the highest in more than a decade and come at a time of economic uncertainty, prompting many employers to question the value of these expenditures.
At the same time, a growing body of evidence suggests that a significant percentage of the money that employers and other purchasers are spending on health care pays for poor quality in the form of overuse, misuse (e.g., medical errors), and waste (Midwest Business Group on Health et al., 2002; Kohn et al., 2000). For example, according to a study by the Midwest Business Group on Health, the Juran Institute, and The Severyn Group, a conservative estimate of the direct cost of poor quality care for employers would be $1,350 per employee per year, while the indirect cost of poor quality care, including lost time and productivity, is at least $340 per employee per year (Midwest Business Group on Health et al., 2002).
Despite the level of activity across the country, the impact of VBP activities on health care quality and costs has not been well established. The evaluation of these activities is a critical step for purchasers eager to identify and adopt beneficial tactics and avoid those found to be ineffective.
Employers Are the Nation's Biggest Purchasers
About 153 million American workers and their dependents receive health insurance as an employment benefit (Gabel et al., 2001). Although some analysts regard the connection between employment and health insurance as an historical accident or question the wisdom of tying health insurance to employment (Battistella and Burchfield, 2000), the fact remains that, collectively, public and private employers are the largest purchasers of health insurance (and consequently health care) in the United States. On average, employers spend $4,164 per employee on health benefits (Battistella and Burchfield, 2000).
Purpose of the Guide
This guide has two primary objectives:
- To encourage purchasers, and especially employers, to conduct formal evaluations of their VBP activities.
- To facilitate that effort by presenting an evaluation process that purchasers can adopt and adapt to their projects.
For many value-based purchasers, one barrier to conducting an evaluation has been the lack of a resource that combines the formality of scientific research principles with real world examples and illustrations. This guide is intended to fill that gap with a tool that is accessible and useful to those without research experience but still informative for researchers.
Another important barrier to both the implementation of value-based purchasing principles and the evaluation of value-based purchasing activities is the fact that most purchasers are too busy with their own initiatives to focus on sharing any knowledge they have gained with other purchasers. To the extent that this guide helps to establish a common framework for evaluating value-based purchasing initiatives, it may serve to encourage the documentation and sharing of knowledge so that in the future, purchasers can learn more from others instead of having to "reinvent the wheel."
Finally, this guide is intended as a resource to be used and shared by decisionmakers as well as those who work on their behalf, whether contractually or otherwise. Specifically, it has been designed so that a senior-level health benefits manager with little research experience can learn about the issues and select a specific analytical strategy without knowing exactly how that strategy might be executed in practice. The manager can then consult with internal or external analysts, who will also find this guide useful for understanding the important analytic, data collection, and measurement details involved in executing the chosen research design. In this way, the guide can help to fill a void that often prevents formal evaluation of value-based purchasing activities from occurring, namely the disconnect between those responsible for making timely business decisions and those experienced in conducting research.
Organization of the Guide
In the first part of the guide, readers will find the following:
- A formal definition of value-based purchasing.
- A discussion of the goals of value-based purchasing.
- Examples of the kinds of activities associated with value-based purchasing.
- A call for well-designed evaluations.
The second part walks through the five major steps involved in evaluating VBP activities.
- Step 1: Define your VBP activities and their goals.
- Step 2: Determine the necessity, appropriateness, and feasibility of an evaluation.
- Step 3: Choose a research design to assess the impact of VBP activities.
This step includes a detailed review of several different research methods that purchasers may want to consider.
- Step 4: Implement the research.
- Step 5: Summarize the results and interpret implications for purchasing activities.
By following these steps, purchasers will be able to develop much-needed evidence about the effectiveness of various VBP initiatives.
Citations for publications that provide further detail on the topics addressed in this guide as well as specific examples of purchasers engaged in VBP activities may be found in the bibliography.
The Basics of Value-Based Purchasing
This section reviews the purpose of value-based purchasing, common goals of VBP initiatives, the kinds of activities that purchasers pursue, and the reasons for a greater emphasis on evaluation.
What Is Value-Based Purchasing?
While there are different ways to define value-based purchasing, at its broadest the term basically refers to any purchasing practices aimed at improving the value of health care services, where value is a function of both quality and cost. It can be helpful to think about value as the result of quality divided by cost:
Value = Quality ÷ Cost
This equation shows that value increases as quality increases, holding expenditure constant.
For the purpose of this guide, value-based purchasing emphasizes activities that aim to improve the quality of care that patients and other consumers of health care services receive. It does not emphasize the various strategies that purchasers use to reduce their costs, even if they are holding quality constant. While many purchasers adopt VBP strategies in an effort to lower their expenses in the long term, it is important to recognize that, although improvements in quality can and often do reduce costs, they may also increase costs or be cost-neutral.
Why Be a Value-Based Purchaser?
As the definition suggests, the main reason to practice value-based purchasing is to get more for your money. On a practical level, this means that your VBP activities should be geared towards influencing one or more final or intermediate outcomes.
One Definition of Value-based Purchasing
"The concept of value-based health care purchasing is that buyers should hold providers of health care accountable for both cost and quality of care. Value-based purchasing brings together information on the quality of health care, including patient outcomes and health status, with data on the dollar outlays going towards health. It focuses on managing the use of the health care system to reduce inappropriate care and to identify and reward the best-performing providers. This strategy can be contrasted with more limited efforts to negotiate price discounts, which reduce costs but do little to ensure that quality of care is improved."
Reason 1: To Improve Final Outcomes
For the purposes of this guide, final outcomes are the results that purchasers ultimately care about: health status, satisfaction with health plans and care delivery, costs, and, for purchasers that are employers, the ability to compete in the market for labor. This section reviews the specific goals associated with these outcomes.
Improved Health Status. Since the primary reason to pursue a VBP strategy is to improve quality, one of the most important outcomes involves changes in the health status of individuals and, in some cases, communities. Ideally, purchasers gauge these changes by evaluating both clinical measures of health status as well as measures that reflect the patient's perspective. The challenge is to be realistic about what impact VBP activities can have on health status and to identify other factors that may influence your results.
Greater Satisfaction With Health Plans and Care Delivery. Most purchasers regard increased satisfaction with health plans and health care as an important final outcome, if for no other reason than to remain competitive in the market for labor. While measures of enrollee or patient satisfaction are not necessarily a reliable reflection of clinical quality, they are easier to assess and understand and are often reported as a proxy for quality.
Lower Costs. For most purchasers, a primary goal of VBP activities is to reduce expenditures associated with health care. Typically, purchasers focus on their own costs, whether measured by premiums or by payments to providers. Some look at costs more broadly by including the initiative's financial impact on the company (e.g., savings achieved by increased productivity or reduced absenteeism). It is also possible to consider savings to patients and their families; for example, an initiative to improve care for asthma may reduce the need for frequent emergency room visits, which impose a measurable cost on families in terms of time and money. Finally, since many VBP activities have an impact beyond an employer and its employees, an evaluation could also attempt to assess changes and shifts in costs at the community level.
Greater Competitiveness in the Labor Market. The many health care purchasers that are employers offer health care benefits as part of the total wage and compensation package for employees. Employer-sponsored health care coverage is the norm in this country for those of working age (18-65) and their dependents and is seen by many employers as a necessity for attracting qualified labor. To the extent that they change the nature of the health benefits package, VBP activities can be related to labor force outcomes such as employee turnover, wages, and the ability to hire new employees (Dowd and Finch, 2001). For example, an employer that significantly reduces its subsidization of employment-based health insurance (and requires more cost-sharing of employees) might experience increased turnover as employees search for jobs with more generous health benefits. While most VBP activities will not have an impact on labor market outcomes, any activities that add, reduce, or alter benefits for existing and new employees might have implications for the employer's competitiveness in the labor market.
Reason 2: To Improve Intermediate Outcomes
Unfortunately, making a direct connection between VBP activities and final outcomes is not always easy. For example, differences in the health status of patients and populations are often difficult to measure; even when they can be detected, there are many other factors aside from VBP activities that could have influenced the change. As a result, you often need to infer the effects of VBP activities from intermediate outcome measures.
Generally speaking, intermediate outcomes are measurable results that have been shown to influence final outcomes. In one study, for example, performance on intermediate outcomes measures predicted the impact of selective referrals to hospitals on mortality rates, an important measure of health status (Dudley et al., 2000). However, these kinds of measures do not determine final outcomes (i.e., an improvement in intermediate outcomes will not guarantee better final outcomes.)
This section discusses some important intermediate outcomes: the selection of high-quality plans and providers by consumers, the utilization of health care services, the prevalence of healthy behaviors, and medical errors.
More Consumers Choose High-Quality Plans and Providers. A common objective of VBP activities is to encourage beneficiaries, patients, or enrollees to select high-quality health plans and providers (e.g., hospitals, medical groups, nursing homes). The theory is that if individuals can identify and choose high-quality providers and plans, they are more likely to experience improvements in their health status.
More Appropriate Utilization of Health Care Services. Many VBP activities focus on reducing inappropriate utilization (e.g., unnecessary Caesarean sections, antibiotic prescriptions for viral infections) and/or increasing appropriate utilization (e.g., compliance with recommended immunizations and preventive care screenings such as mammograms). There is ample evidence linking the appropriate utilization of services to better health status as well as lower costs, but the determination of appropriateness is not always straightforward.
More Evidence of Healthy Behaviors. Many purchasers, especially employers, change their benefits package or collaborate with health plans and providers in an attempt to influence the health-related behaviors of enrollees and patients, such as smoking, alcohol use, and exercise. A change in the prevalence of these behaviors is a relevant intermediate outcome because of its direct relationship with costs and health status. For example, smoking, excessive alcohol consumption, and other inadvisable or risky behaviors are related inversely to health status and positively to health care expenditures. Similarly, exercise is related positively to health status and inversely to health care expenditures. Thus, to the extent that these programs are successful, they would likely be correlated with better health status and lower expenditures.
Fewer Medical Errors. Some prominent purchasers are beginning to respond to recent concerns about patient safety by developing VBP programs aimed at minimizing medical errors (Leapfrog Group, 2001). Medical errors include errors of omission (e.g., a failure to diagnose a health care problem requiring treatment) as well as errors of commission (e.g., a surgery performed on the wrong knee, an overdose of an appropriate medication). While this is a new focus for value-based purchasing, initiatives that lead to a reduction in health care mistakes are expected to improve quality and lower overall costs.
What Do Value-Based Purchasers Do?
Because quality is a broad concept with many dimensions, value-based purchasing encompasses a wide range of initiatives designed to achieve a variety of short-term and long-term objectives. Table 1 lists a number of purchasers engaged in VBP activities; the studies cited in this table provide useful information about the specific VBP activities in which these purchasers are engaged as well as, in some cases, insights into the impact of those efforts. This literature suggests that VBP activities generally focus on three groups:
- Those who are eligible for or receive health care (e.g., employees, patients).
- Those who provide health care (e.g., health plans, physicians, hospitals).
- The third parties who pay for health care (e.g., insurance companies).
Despite the variety at the tactical level, there are essentially two paths or strategies that purchasers can follow to influence final outcomes. Strategy 1 focuses on influencing the decisions or behavior of individuals (i.e., employees, beneficiaries, or patients), while Strategy 2 aims to change the behavior or performance of health care entities, usually providers and/or plans. Many large purchasers pursue both strategies at the same time, but smaller ones often lack the resources to take on multiple activities.
Watch Out for Unintended Consequences
When you conduct an evaluation, it is critical to identify the intended consequences of a VBP activity so that you can measure and monitor the appropriate final and intermediate outcomes and determine whether a relationship exists between the VBP activity and these outcomes. However, VBP activities sometimes result in outcomes that were not intended or predicted. Since these unintended consequences could undermine your broader goals, it is important to seek out and assess them as part of your evaluation of the overall impact of a VBP program, even though the evaluation may not be designed for that specific purpose.
For example, imagine a VBP activity that rewarded providers that had low mortality rates for CABG (coronary artery bypass graft) surgery. Providers might achieve these goals by genuinely improving quality of care. However, such initiatives might also discourage providers from treating the sickest patients, an outcome of value-based purchasing that is not desirable.
Select for Figure 1 (31 KB).
Strategy 1: Change the Behavior and Decisions of Individuals
The first strategy is to encourage people to make choices that will lead to higher quality care and better health. While the primary goal of this approach is to affect the health care-related decisions of consumers, this market-oriented strategy has an implicit objective to change the behavior of health plans and providers, which would be expected to improve their performance in order to attract enrollees and patients.
At a tactical level, this would include activities such as consumer information campaigns (e.g., general education about health care quality, the distribution of specific data on the performance of providers or health plans), as well as the use of financial incentives or cost sharing to encourage the selection and use of providers and health plans that can document their ability to provide high-quality care. For example, based on evidence that mortality rates as well as other measures of quality are positively related to surgical volume, some employers use selective contracting or incentives to encourage employees to go to hospitals that perform a large number of surgical procedures.
To assess the impact of this kind of activity, you could look at how many employees are receiving surgery at high-volume facilities or enrolling in highly rated health plans. A more involved evaluation could track the impact of this strategy on quality and costs and identify any unintended consequences.
Initiatives designed to support this strategy, such as programs to educate and inform consumers about quality, are often the first choice of relatively smaller purchasers (e.g., medium-size employers) that lack the market clout needed to deal directly with providers and plans. Among larger employers, some adopt this "consumer-empowerment" strategy because they regard it as consistent with their human resources philosophy. Others prefer it because they want to maintain an arms-length relationship with the business of health care. However, it is important to recognize that, compared to Strategy 2, this strategy has a less direct impact on the delivery of care because it depends on the ability of consumers to drive changes in the health care market.
Examples of Ongoing VBP Initiatives To Inform Consumers
Reporting Health Plan Quality: The AboutHealthQuality Web Site
As part of its VBP activities, the New York Business Group on Health is a leader of the New York State Health Accountability Foundation, a public-private partnership co-founded by IPRO (an independent quality evaluation organization) and funded in part by New York State. The Foundation sponsors the AboutHealthQuality Web site, which offers consumers comparative information on the performance of health plans throughout the New York metropolitan area. For more information, access www.abouthealthquality.org.
Reporting Hospital Quality: The Hospital Profiling Project
A hospital project initiated several years ago by Ford Motor Company in Southeast Michigan has evolved into a multi-purchaser, five-city initiative to collect data and report on the quality of inpatient care. Participating employers distribute the results to employees and retirees through publications and Web sites. For more information, access www.hospitalprofiles.org.
Reporting Medical Group Quality: The Consumer Assessment Survey
For a number of years, the Pacific Business Group on Health (PBGH; a coalition of large West Coast purchasers) has measured and reported on the quality of care delivered by medical groups in California. Using the Physician Value Check Survey, and more recently the Consumer Assessment Survey, PBGH has produced annual public reports with measures of consumer satisfaction and the quality of preventive care at the level of group practices. For more information, access www.healthscope.org.
Reporting Health Care Quality: The TalkingQuality Web Site
In March 2002, AHRQ, the Centers for Medicare & Medicaid Services (CMS), and the U.S. Office of Personnel Management launched a new Web site, TalkingQuality.ahrq.gov, that provides easy-to-use information on health care quality. The site is designed for organizations and professionals who are experienced in producing quality reports for a variety of audiences, as well as those who are not. Detailed information is found within the site about the entire process of communicating information from the initial conceptualization of the idea through the project's implementation and, finally, the evaluation phase. For additional information, visit the site at https://talkingquality.ahrq.gov.
Strategy 2: Change the Performance of Health Care Organizations and Practitioners
The second strategy for improving intermediate and final outcomes is to effect changes in the performance of health care organizations and practitioners. In their study of the pioneers of VBP, Meyer et al. (1997) identified four types of organizations that VBP activities target for purposes of changing provider behavior or performance: health plans, health care systems, hospitals, and physician groups. Activities include:
- Standardizing benefits across health plans in order to facilitate apples-to-apples comparisons of value.
- Requiring that providers be accredited.
- Encouraging plans or providers to adopt specific disease management programs intended to improve health outcomes.
- Requiring that health plans report measures from the Healthplan Employer Data and Information Set (HEDIS®) and/or the Consumer Assessment of Health Plans (CAHPS®). (Select for information on HEDIS® and CAHPS®.)
- Requiring that hospitals report mortality or complication rates.
- Monitoring these reports to identify areas in need of improvement.
- Incorporating quality standards into contracts with health plans or care systems. (contractually linked groups of primary care physicians, specialists, and hospitals)
Activities focused on third-party payers, including contracting with Preferred Provider Organizations (PPOs) to obtain discounted health care fees from physicians, hospitals, and other health care providers, are considered to be value-based purchasing only if quality is a key component in the contracting.
As with strategies directed at individuals, the measurement of the direct impact of these programs on final outcomes such as health status is often difficult. It is usually more feasible to link these types of VBP activities to intermediate outcome measures that have been shown to be associated with final outcomes, such as improvements in the percentage of diabetics screened for potential complications or the percentage of women screened for breast cancer.
Example: The Leapfrog Group's Quality Standards
With a membership of about 90 health care purchasers, the Leapfrog Group is a national organization committed to reducing medical errors and improving the value of health care. To improve patient safety, Leapfrog members are trying to change what providers do by insisting that hospitals implement the following practices:
To evaluate their success, the members plan to look at intermediate outcome measures such as the change in the percentage of a health plan's network hospitals that have electronic order entry systems or proper ICU staffing (Leapfrog Group, 2001). A more complex evaluation would track impact of referral policy on health status and plan costs. (Leapfrog members are also pursuing the first strategy described above by initiating efforts to educate consumers about consumer safety.) For more information, visit http://www.leapfroggroup.org.
A Goal Within Each Strategy: Reduce Imbalances in Information
While many VBP activities are intended to have a direct influence on the behaviors or decisions of consumer or health care organizations, other VBP activities take an indirect approach by trying to reduce or eliminate asymmetric information between health care providers and plans on the one hand, and purchasers and consumers on the other. In a perfectly informed market, organizations and individuals would be able to select plans and providers that correspond to their preferences for quality and costs. However, asymmetric information poses a barrier to improving the final outcomes that purchasers are concerned about. To the extent that VBP activities reduce information asymmetries, they should improve the functioning of market mechanisms and lead to intermediate and final outcomes that more closely satisfy organizational and consumer objectives.
For example, some large purchasers issue requests for proposals (RFPs) or requests for information (RFI) that require health insurance plans (or care systems) to bid on standardized benefit packages. While this tactic may not have a direct impact on the decisions of providers or consumers, it does provide the purchaser with comparative information that it can use to influence those decisions and, ultimately, improve value. For instance, if you could evaluate variations in premium quotations and quality provisions across competing plans offering standardized benefit packages, you would be in a better position to identify, contract with, and offer employees incentives to select the plans or care systems that offer the best price and quality.
The V-8 Initiative: A Concerted Effort To Reduce Asymmetries in Information
In 1997, a group of eight regional employer coalitions decided to develop a standardized RFI that would help employers and other purchasers identify and select high-quality health plans. This group, called the V-8, was recently joined by General Motors and Marriott International; other participants include accrediting bodies and government agencies, including the Centers for Medicare & Medicaid Services.
The standardized RFI, which has been in use for 3 years, makes it easier for purchasers to gather useful data on their health plans and make apples-to-apples comparisons. The V-8 participants also regard it as a tool for leveraging employer clout to enhance quality. In addition, the RFI benefits health plans and provider systems because it should eventually reduce the variety of information requests they receive from purchasers.