Toward a Research Agenda on Quality-Payment Alignment
Findings From an Invitational Colloquium
On November 9, 2006, the Agency for Healthcare Research and Quality (AHRQ) and the Commonwealth Fund convened a colloquium to develop a national research agenda on aligning quality with provider payment incentives and consumer financial incentives. This document summarizes the colloquium findings and recommendations for research to investigate the effect of these incentives on improving the quality of health care.
Despite steady growth in health care spending, both private and public payers have become increasingly dissatisfied with the care they are buying. Although the current health care financing system encourages the provision of more care, it does little to ensure that individuals receive appropriate care or that the care they receive is effectively or efficiently provided. As a result, payers have, in recent years, implemented an array of strategies aimed at using financial incentives to promote higher quality care, with the expectation that this will lead to a better return on their spending; these strategies have been aimed at both providers and consumers.
However, despite widespread interest in these strategies, there is a lack of concrete evidence to support their effectiveness.1 Presuming that effective approaches for using financial incentives to improve care can be found, more information is needed on the best ways to design and implement those approaches in different situations. Although some research is being done on the alignment of payment incentives with quality, critical gaps in our collective knowledge base call for prompt attention from the research community, including funders of research.
To clarify the need for further research and define areas for investigation, the Agency for Healthcare Research and Quality (AHRQ) and the Commonwealth Fund convened a colloquium on November 9, 2006 to develop a research agenda on aligning payment with quality. The colloquium addressed two related major gaps in knowledge:
- Evidence related to the impact of payment mechanisms that reward health care providers for improving quality and ways to develop effective mechanisms that apply to different types of providers.
- Evidence on financial incentives aimed at rewarding patients—the consumers of health care services—for choosing high-quality providers.
In addition to recommending topics for research specific to provider and consumer incentives, the colloquium also examined these issues:
- Related priorities, including the need to expand the existing library of performance measures available to purchasers.
- Dissemination strategies to enhance the uptake of best practices.
- The lack of "systemness"i in the U.S. health care system and consequences for quality performance.
Research has suggested that the complexity of health care financing and delivery structures can lead to delivery inefficiencies and variations in quality and safety.2 A better understanding of the determinants of systemness and the comparative advantage it offers in terms of quality performance is needed. More broadly, colloquium participantsii recognized that initiatives to align payment with quality must take place within a larger effort to make health care payment mechanisms more effective and efficient. A robust research agenda can benefit not only purchasers, but also providers and consumers—who have a considerable stake in the success of initiatives that align payment with quality.3
i. "Systemness" is defined here as the capacity of provider organizations to coordinate health care across providers, settings, and conditions in order to best address the needs of the patient and the population. When applied effectively, care management, patient education and support, and the use of clinical information systems may promote systemness.
ii. Go to the Appendix for a list of colloquium participants.
Provider Payment Incentives
The Federal Government as well as many organizations and purchasers have embraced pay-for-performance approaches as essential to meeting quality improvement goals. Examples of these national initiatives include:
- The Leapfrog Group's Hospital Rewards Program recognizes and rewards hospitals for their performance in both the quality and efficiency of inpatient care.
- Bridges to Excellence, a multi-State, multi-employer initiative, has designed programs that reward physicians for high performance related to diabetes and cardiac care.
- The Centers for Medicare & Medicaid Services (CMS) is developing and testing ways to tie Medicare hospital and other provider payments to quality of care measures.
- The President's recent Executive Order 13410, "Promoting Quality and Efficient Health Care in Federal Government Administered or Sponsored Healthcare Programs," calls for collaborative efforts to promote quality and efficient delivery of health care through the use of better incentives for health care providers and consumers.
In addition to these national initiatives, many regional purchasers are pursuing programs that align payment with quality. Yet, despite the growing popularity of pay-for-performance initiatives that reward providers, there is little definitive evidence on how best to design these programs and on whether these mechanisms achieve the desired result—improved quality of care.
In a comprehensive review of literature and ongoing research, Dudley and colleagues found evidence that performance-based payment can work in certain situations, but data to help purchasers in designing such payment strategies were limited.4,5 The researchers examined the results of eight randomized controlled trials of performance-based payment. Those trials tested variables such as:
- Who receives the incentives.
- The magnitude and structure of the incentive.
- Whether the incentive targets preventive or chronic care.
- Other factors such as the degree of patient cooperation.
Results of the evidence review were mixed. For example:
- In one study, residents at a university-based pediatric clinic who received a $2-per-visit case management fee did better complying with well-child care recommendations than did a control group that did not receive any additional compensation.6
- In another study, a randomized clinical trial in Philadelphia found that capitated medical groups that received bonuses ($570 to $1,260 per site) for increased rates of compliance with mammography, Pap smear, and colorectal screening performed no better than a control group that received no bonuses.7
These results suggest that the ability of an incentive to improve performance rests on a number of different factors, some of which are characteristics of the incentive itself and others are external to the incentive. Patient characteristics may also play a role.
Characteristics of the Incentive
Important characteristics of the incentive include, for example, its compatibility with the underlying payment system and whether the recipient of the reward is an individual provider or group. Incentives must target the individual or organization that is in control of the performance being measured. For example:
- Performance under the control of the physician—such as ordering unnecessary tests—should be addressed with incentives that target the individual physician.
- Performance problems that are linked to systemic weaknesses, however—such as medication errors that result from inadequate information systems—demand system-oriented incentives.
Potential revenue to be gained from the incentive and costs associated with compliance are additional factors that affect the ability of an incentive to improve provider performance. Rewarding high-performing providers with publicity may contribute to the success of financial incentives.
External factors include the presence of other incentives in the market and characteristics of the provider organization.4 Rewarding providers for meeting certain quality goals helps to align quality with payment, but providers may face obstacles in achieving those targets. Strong information technology capacity and an effective communication style within a provider group may contribute to the success of a financial incentive.
Additional resources to establish a solid infrastructure may prove essential in enabling providers to improve their performance and in ensuring that purchasers reap the full benefit of pay-for-performance programs. In Medicare, Quality Improvement Organizations (QIOs) are required to make these additional resources available to providers, but further monitoring and research is needed to determine the effectiveness of QIOs in carrying out this function.8
Finally, patients themselves play a role in the success of efforts to align payment with quality. Patients' education and willingness to take on self-care can influence a provider's ability to achieve a performance target. The nature of the interaction between patient and physician may also influence a provider's performance against quality measures. Making additional payment available so that the provider can serve as the patient's 'medical home' by coordinating care for the patient across settings may encourage more effective and efficient delivery of services, avoiding unnecessary complications and costs.8
These findings suggest that aligning payment with quality as a strategy to improve the performance of providers can work, but it must take into account key contextual factors. Furthermore, the randomized controlled trials examined by Dudley and colleagues focus on the performance of a single indicator or aspect of care. These findings have limited applicability to the current reality of pluralistic purchaser efforts that tackle multiple aspects of care.9 Without further evidence to support these promising but preliminary findings, purchasers risk investing time and resources on strategies that will not produce the desired return both in terms of quality improvements and cost-effectiveness.4
Consumer Financial Incentives
Research on the impact of financial incentives on consumer health behaviors is sparse. Despite some experimentation with consumer incentives to encourage uptake of certain preventive services, there have been few applications that reward consumers' selection of high-quality services.
Consumer-oriented financial incentives may be designed to encourage selection of high-quality providers, promote certain health behaviors, or increase adherence to treatment protocols. Examples may include the following:
- Lower copayments.
- Incentive schemes.
- Contingent payments that remove financial barriers.
A few studies indicate that financial incentives directed at consumers can boost patient compliance, improve immunization rates, and enhance positive health behaviors.10 Giuffrida and Torgerson synthesized the results of 11 randomized trials that examined the use of financial incentives to increase adherence to medical treatments—specifically, compliance with medication, medical advice, and medical appointments.11
In 10 of the 11 studies, there were improvements in patient adherence associated with the use of financial incentives. Most of these studies, however, had relatively small sample sizes, and no study compared the impact of using different levels of financial incentives. Furthermore, none of the 11 studies addressed the relative effectiveness of cash versus payment in kind as an incentive.
Kane and others conducted a literature review of previous studies examining the effect of financial incentives on motivating consumers to adopt or maintain simple or complex preventive health behaviors.12 After reviewing 47 articles—24 related to simple preventive care and 23 to more complex care—the authors concluded that economic incentives appeared effective in encouraging patients to improve their approach to preventive health.
Furthermore financial incentives were effective about the same percentage of time for simple (74 percent) and more complex (72 percent) preventive health behaviors. Even small incentives were effective in changing behavior, although larger cash incentives led to larger behavioral responses. Findings from this literature review, however, left unanswered the important question of how large an incentive is needed to sustain a complex behavior change.
Published research does not support the effectiveness of financial incentives directed at smoking cessation programs. Hey and Perera examined 15 randomized clinical trials or studies and determined that significantly higher quit rates were not achieved for any of the incentives groups when compared to control groups.13 In fact, people who committed their own money to be part of a smoking cessation program did no better than those who did not.
Although these studies do provide insight on how financial incentives may be effective at increasing positive health behaviors, they raise additional questions:
- How can consumer incentives be used—alone or in tandem with provider incentives—to improve quality of care?
- How would consumer response vary by the size of financial incentive?
- Are financial incentives worthwhile, given the cost of paying for the incentives themselves and then marketing them?
Additional research would improve our understanding of how financial incentives could help engage patients in selecting high-performing providers.
A National Research Agenda
Numerous initiatives are under way both to remunerate providers for quality performance and to encourage consumers to choose high performing providers by giving them financial incentives. However, the literature on the performance of these mechanisms is limited. As a result, a coherent research initiative has significant potential to inform both current and future efforts to align payment with quality. A national research agenda must focus on these areas:
- Effectiveness of financial and other incentives in rewarding high-performing providers.
- Effectiveness of consumer-directed financial incentives to reward and encourage selection of high-quality providers.
- The interplay of provider and consumer incentives and how to apply these approaches to public programs—Medicare and Medicaid—given their unique program dynamics.
Provider Payment Incentives
Colloquium participants were particularly interested in the potential for applying pay-for-performance strategies to encourage systemness and improved team performance. At the same time, research is needed to develop ways to encourage increased efficiency and equity. Purchasers may also wish to use incentives to improve care to underserved populations—both demographically and geographically underserved—thereby effectively offering rewards for reducing disparities in quality and access to care.
Researchers must focus on producing a better understanding of payment incentives aimed at providers, whether hospitals or physicians—either solo practitioners or groups of providers—or other clinicians. A national research agenda aimed at aligning payment with quality must address the gaps in our understanding of provider payment incentives outlined below.
How can incentives be used to encourage broader collaboration and coordination across systems? Performance measures used by purchasers tend to conform to existing silos of care. Participants expressed interest in developing measures that would encourage shared data and coordinated care across treatment settings in an effort to avoid unnecessary complications and costs. Research is needed to determine whether payment incentives can change patterns of referral and communication among providers to encourage such coordination and systemness, particularly in systems where patients are free to obtain care from any available sources. Such an approach may require the development of new quality and efficiency measures that could serve as the basis for rewarding types of individual provider for care coordination across treatment settings.
Can we design effective performance measures that reward teams of providers and integrated groups? Participants identified the need for performance measures that encourage teams of clinicians to provide coordinated, efficient, and evidence-based care. Research is needed that would examine possible measures of system—rather than individual provider—performance and incentive approaches that can be applied to teams of providers, such as systems of hospitals and other facilities, medical groups, and other groups of clinicians. These incentive approaches would encourage systemness by paying bonuses to more integrated provider groups.
Are penalties or rewards more effective? Research to date is not clear as to whether providers are more likely to be motivated by threats such as loss of income or rewards such as cash bonuses. Several pay-for-performance programs, including the CMS Premier Hospital Quality Incentive Demonstration, incorporate penalties for poor performance. More recently, purchasers are exploring new reward mechanisms such as gain sharing, or shared savings, whereby providers and payers share the savings resulting from improved quality and efficiency. Research on the impact of these approaches on provider performance and quality of care is needed.
Studies are also needed that examine the effects on quality of combination approaches—those that integrate financial rewards or penalties with reputational incentives that reward high-performing providers with publicity. The role of infrastructure to support performance improvements (system standards and specifications, technical assistance, etc.) also needs to be investigated further.
What factors contribute to sustainability of quality-based payment initiatives? Research questions arise on the sustainability of these initiatives as well as factors associated with their success. Many purchasers wish to coordinate with a wider group of stakeholders, but some anecdotal evidence suggests that such collaborative efforts may be difficult to sustain over the long term. Yet as pay-for-performance programs mature, successful collaboration among stakeholders may prove essential. Further research could shed light on factors that contribute to the longevity of programs that have successfully aligned payment with quality.
What additional performance measures are needed for preventive services and specialty care? Purchasers interested in aligning payment with quality have a relatively limited set of performance measures from which to choose. While purchasers pay for a broad array of clinical services, the available set of performance measures does not represent all types of care. Although substantial efforts in this regard already are underway, a national research agenda focused on provider payment incentives must coordinate and encourage more efforts to develop and improve the sets of performance measures available to purchasers and providers.4
Valid, reliable measures are a prerequisite for pay for performance and some specialties and subspecialties need research that supports development of measures. In particular, measures are needed for certain types of specialty care and for care coordination. Furthermore, most measures have focused on preventive care and structure or processes rather than outcomes.
How many measures are enough? Pay-for-performance strategies have helped fuel a demand for performance measures. It is important to understand the number of measures that are best targeted in pay-for-performance initiatives. Some programs use as few as 4 measures in determining performance rewards (or penalties) while others, such as the new general practitioner contract in the United Kingdom, use over 100.
What performance measures are needed for provider categories beyond hospitals and physicians? Finally, most pay-for-performance efforts have targeted physicians, hospitals, and managed care plans. Colloquium participants acknowledged that further research is needed on the effectiveness of such strategies when applied to other provider categories, including nursing homes, home health, dialysis, rehabilitation, and mental health providers to name just a few.
Consumer Financial Incentives
Most of the research on consumer engagement or activation has focused on eliciting positive patient behavior through a variety of strategies but has largely ignored the use of financial incentives. Few rigorous studies of the impact of financial incentives on consumer health behaviors exist in the published research. Increasingly purchasers are choosing to make transparent to consumers information on the quality and price of services; this trend has fueled interest in consumer incentives as a means of supporting the effectiveness of this approach. A national research agenda on aligning payment with quality must address the following questions about how patients respond to monetary incentives.10
What is the impact of consumer incentives on provider selection and tiered networks? Research is needed on the impact of coupling consumer financial incentives with high quality provider designations. Or, more broadly, how might we use financial incentives to influence choice of provider? Additional research on consumers' experiences with tiered networks and patterns of provider selection in these networks would help inform the field.
How much of a financial incentive to support complex behavior change is enough? Existing research provides little information about how consumer response varies with the size of financial incentives. Furthermore, research is needed to examine the marginal effect of financial incentives when coupled with other types of consumer interventions.
Are consumer financial incentives cost effective? Few studies explore the cost-effectiveness of incentive programs, taking into account the cost of marketing and of the incentive itself in relation to the desired outcome.
Many current transparency initiatives can be evaluated for their impact on system performance. For example, the Department of Health and Human Services (HHS) is supporting a network of local collaborative organizations that are working to improve quality and value in health care by assessing the performance of local health care providers and reporting these findings publicly. Under the plan, HHS will select qualified regional collaboratives, referred to as Value Exchanges, which will be encouraged to implement provider and consumer incentives along with public report cards.
In addition, the Robert Wood Johnson Foundation recently launched its Aligning Forces for Quality program with 14 pilot projects. This initiative is designed to help communities advance the quality of chronic care by:
- Working with providers to improve the quality of ambulatory chronic illness care.
- Measuring and publicly reporting the performance of the providers.
- Promoting consumer demand for higher quality care.
These quality initiatives make them important in determining the extent to which transparency can in fact both motivate and help consumers to select high performing providers. These initiatives also offer a selection of natural experiments that can be evaluated and compared in the future.
Interplay of Provider and Consumer Financial Incentives in Public Programs
There are many efforts in the public sector to design payment systems that will lead to improvements in the quality of care. A recent Institute of Medicine report proposes a plan for applying pay for performance in the Medicare program.14 This report provides a platform for further research that can make a substantial impact on current and future initiatives in both the private and public health care systems.
Many States are either implementing or considering pay-for-performance approaches in their Medicaid programs despite regulatory and budgetary hurdles, and others are exploring the use of consumer incentives. Florida's new Medicaid reform plan is an example of a State that is experimenting with consumer financial incentives to encourage healthy practices.
In 2006, Florida began implementing a plan that enables Medicaid participants in two counties to choose among a variety of private-sector managed care plans offering benefit packages tailored to their needs. In addition to comprehensive and catastrophic benefit packages, the plans are offering an enhanced benefits package with an incentive for those Medicaid participants who engage in healthy behaviors. Participants who, for example, join a weight loss program or participate in a smoking cessation program can collect up to $125 annually in a spending account. Participants can apply the credit in their account towards purchase of over-the-counter health-related items such as vitamins.
These national and State initiatives suggest a number of research needs that could improve our understanding of the effects of financial incentives on providers and consumers in public programs.
Paying for performance in Medicare. CMS and Congress have both expressed interest in applying provider payment incentives in Medicare. Pay-for-performance initiatives in the private sector have the potential to offer guidance to CMS as it explores alternative payment arrangements.14 CMS has already begun to invest considerable resources in pay-for-performance efforts to support quality improvement in the care of Medicare beneficiaries.
In addition to pay-for-performance initiatives for hospitals, physicians, and physician groups, CMS is exploring opportunities in nursing home care and is considering approaches for home health and dialysis providers as well. CMS is also pursuing pay-for-performance strategies to support better care coordination for patients with chronic illnesses. Evidence on provider payment incentives—what works and for whom—could help CMS as it develops these new initiatives and would increase their likelihood of success.
Effectiveness of quality-based payment in Medicaid. Evidence on how to deal with the unique circumstances that rewarding providers in safety net settings may present is limited. Further research is needed to explore challenges of quality-based payment in Medicaid programs, in which patients may be sicker, quality and information infrastructures may be looser, and the perspectives of providers may be different from those of their private-sector counterparts. Many State Medicaid programs are also challenged by a lack of participating physicians. These programs often face budgetary and regulatory constraints that may affect their approach to quality-based payment.15
Patient financial incentives in Medicare and Medicaid programs. Little research has been undertaken to determine the effectiveness and feasibility of financial incentives aimed at Medicare beneficiaries and Medicaid recipients. Medicaid programs face unique challenges in terms of adherence in patient populations with low literacy and socioeconomic challenges. In the Medicare program, research examining whether financial incentives such as lower copayments or premium deductions would work for seniors is also needed.16
Purchasers urgently need research findings and information on best practices that have immediate, practical applications. They need to know whether an intervention works and what its benefits and costs are. They want to know if a "business case" exists.
To achieve this practicality, the research agenda proposed here faces some real, but surmountable, challenges. First are a number of important research design considerations. Second, to accomplish the agenda, additional funding sources will be needed. Third, dissemination of findings continues to pose a challenge to a research community not necessarily prepared to meet the practical needs of purchasing executives and other decisionmakers in a timely manner.
Research designs must yield usable and timely findings applicable to purchasers who are in a position to design and implement provider and consumer incentive programs. Payers want information that they can apply right away. They need to know whether a payment mechanism works, how it performs relative to other interventions, and the costs and benefits of implementing it. These practical concerns suggest the need for a tradeoff between rigorous methods and timely research. A research study that takes several years to complete may be irrelevant by the time it is published.3 As a result, although randomized controlled trials (RCTs) make sense for some experiments, in general they are an unwieldy solution. Not only are RCTs costly and difficult undertakings, researchers are often unable to convince providers to participate.5 Research designs with a strong control may offer a potential solution.
The field is in need of support for both rigorous qualitative and quantitative research studies. Planned natural experiments may make more sense because so many incentive programs are already underway. Some of these efforts may have a non-randomly selected control group that could be used for evaluative purposes. The absence of adequate data sets has hindered evaluation of some existing pay-for-performance initiatives.
Additional Funding Sources
Research on aligning payment with quality is being conducted by AHRQ, the Commonwealth Fund, the Robert Wood Johnson Foundation, and the California HealthCare Foundation. Additional efforts, and perhaps additional funders, are needed to take on aspects of the critical research agenda outlined here.
One of the challenges for any research agenda is appropriate and timely dissemination of findings. These findings are aimed at busy purchasing executives who are deluged daily with print and electronic information and need research findings that capture their attention. Furthermore, these executives want information not only on what has worked, but also on the reasons why some past strategies to align payment with quality have failed. Purchasers need to see research that may in fact demonstrate that some pay-for-performance strategies have no impact. Colloquium participants expressed concern that a natural bias against studies that show no impact may prevent publication such findings.
The February 2006 issue of Medical Care Research and Review offered a special supplement that sought to expedite the transfer of research on quality-based payment to public and private purchasers. The research studies in this publication were accompanied by commentary from a range of experts. This model may help facilitate adoption of research findings, particularly when commentary is offered by experts from like-minded purchasing organizations.
More than 100 health plans, employer coalitions, and public insurance programs are engaged in pay-for-performance programs.17 Over 200 examples of health care report cards can be found at https://talkingquality.ahrq.gov, a Web site intended to help those who are providing consumers with information on the performance of health plans and providers. Many of these pay-for-performance and report card efforts are in their early stages. Purchasers and providers involved in these and future initiatives urgently need research findings that are timely, usable, and reflect best practices. The following are among the opportunities recommended for consideration as resources to help purchasers, providers and consumers in applying the evidence:
Attendees identified the need for researchers and purchasers to have a credible resource for the latest research findings and best practices in the field of pay for performance. A dedicated Web site could serve, at least in the initial stages, as a clearinghouse on the latest research and as a forum to bring together researchers and the purchasing community to identify research needs and issues related to implementation of efforts that align payment with quality. Such a Web site would also serve as a reliable means of disseminating studies whose findings show no effect from a particular type of pay-for-performance approach and, as a result, have difficulty achieving publication.
This group would guide further development of a national research agenda on aligning payment with quality and would provide a forum for major public and private funders to showcase their work and learn first hand about the work of others. Advisory group members would represent the leaders of Federal agencies and national foundations that fund this research, as well as members of the research and purchasing community. This group would also inform the activities of the pay-for-performance clearinghouse.
In addition, during the initial stages, a single entity to organize and coordinate these efforts would be useful.
Purchasers and providers will need to continually adapt their payment incentive designs as researchers collect and disseminate evidence to support best practices.1 For the health services research community and the funders who help support them, the challenge is to keep accumulating evidence that supports effective payment and reward strategies, and ultimately, leads to improved quality of care.