Skip Navigation Archive: U.S. Department of Health and Human Services www.hhs.gov
Archive: Agency for Healthcare Research Quality www.ahrq.gov
Archive print banner
Evaluation of AHRQ's Partnerships for Quality Program

This information is for reference purposes only. It was current when produced and may now be outdated. Archive material is no longer maintained, and some links may not work. Persons with disabilities having difficulty accessing this information should contact us at: https://info.ahrq.gov. Let us know the nature of the problem, the Web address of what you want, and your contact information.

Please go to www.ahrq.gov for current information.

Chapter VII. How Significant Overall Was PFQ in Contributing to AHRQ's Broader Strategic Goals?

A. Accomplishments and Limitations

From our perspective, PFQ was reasonably successful as a grant program taking into account the varied objectives of the diverse grantees that were funded. Most grantees did what they said they would, although the overall impact of all 20 projects was not as fully realized as AHRQ program initiators had hoped. 

PFQ had a core set of 12 grants focused on directly changing clinical practice and outcomes, at least 8 of which already have some evidence of positive outcomes. Most of these efforts produced sustainable changes in day-to-day practice that will enable and foster regular quality monitoring and continuous quality improvement in nursing homes, primary care physician offices, hospitals and home health agencies.  While five other projects had goals that also focused on improving clinical quality and outcomes but stopped short of trying to directly change practice, they did generate valuable lessons about how to provide an infrastructure and set of financial incentives for such efforts. The bioterrorism preparedness grants, whose goals were to improve the health system's ability to respond to emergencies, also appear to have generated valuable knowledge.  

For a pioneering program, these accomplishments are impressive.  They provide a foundation of learning that AHRQ can build on for improving the safety, quality and effectiveness of health care delivery. The partnerships created have leveraged resources from national and community-based organizations for promoting improvement, and forged stronger linkages between researchers and those on the front line of health care delivery.

While relatively successful on these metrics, PFQ had some shortcomings.  First, a few of the grants probably were not, with the benefit of hindsight, well-conceived originally, despite their best intentions.  Second, PFQ grantees did not have the scale of impact originally expected by AHRQ's program developers, or promised in the RFA and the program announcement.19  While the grantees' interventions reached a meaningful number of providers, they clearly reached fewer than one would expect solely by the membership of major organizations involved with PFQ (e.g. AMA, AHA etc). However, those initial expectations on the part of AHRQ were probably unrealistic, given the nature of the grants funded and the scale of the projects' goals.  Third, this evaluation suggests that PFQ's efforts to promote collaboration and mutual learning across PFQ grantees through AHRQCoPs and other cross-grantee work was not very successful in supporting grantees, though it may generate some useful publications.

Return to Contents

B. Future Opportunities and Lessons

Particularly because PFQ was an early initiative to support one of AHRQ's current priorities—transforming research into practice—the formal ending of the PFQ program provides an important opportunity to harvest lessons that may be valuable to AHRQ for the future. While AHRQ could expect some failures in a program that aimed to encourage innovation, AHRQ can learn from its experience on how it managed the PFQ program and apply the lessons to current and future initiatives designed to translate research into practice, and to use partnerships to extend the reach of its quality, safety, and security improvement efforts.

To date, little has been done to extract the lessons about what worked well and take advantage of the opportunities they present. The lessons learned about what did not work are equally useful. The initial lessons and findings presented in this report can help AHRQ achieve many of its goals. We review here four of the most significant lessons and insights from PFQ, and offer several avenues for AHRQ to apply the lessons to its current priorities.

1. Elements of Effective Partnerships for Translating Research into Practice

PFQ grantee experiences and lessons can help AHRQ create more effective partnerships for bringing to scale and speeding up the translation of research into practice. Critical elements of effective partnerships, listed at the conclusion of Chapter VI, include: national organizations and individual leaders with expertise and well-regarded reputations in the health care issue or topic of focus, selection of well-connected partners at all levels—grantees, intermediaries and target organizations—and strategic use of each one's resources and connections; skills and experience in partnership management; and the ability to overcome provider barriers to change. 

Partnerships, and how to promote them, are important to many AHRQ programs. For example, the concepts have immediate relevance to the ACTION program, AHRQ's latest effort to use "field partnerships" to translate research into practice. In fact, several PFQ grantees are participating in ACTION networks, offering an opportunity for them to share their own lessons in partnership building with the collaborators in each network.  But other ACTION grantees also would benefit from knowing more about the collective experience and lessons from the PFQ projects to gain insight in fostering teamwork and partnerships. Other AHRQ programs involving partnerships, such as the 17 projects funded by the Partnerships in Implementing Patient Safety (PIPS), may also benefit from learning more about the PFQ experience and lessons. 

AHRQ managers and staff also participate in a wide variety of partnerships with other federal agencies and private health care organizations, from work on CAHPS to leadership of the AQA-HQA efforts to develop standardized performance and quality measures.  Lessons about effective partnerships are potentially applicable to AHRQ's work in these other efforts as well. A work group within the agency might be created to distill the lessons on effective partnership management from these initiatives, and determine how they could be applied to strengthen AHRQ's existing and future partnerships and programs.  

2. Leveraging AHRQ's Internal Resources to Help Translate Research into Practice

The PFQ program provides good examples of the way an effective project officer can help leverage the work of grantees.  Specifically, those AHRQ project officers that brought with them connections and deep knowledge of particular issue areas took the initiative to connect principal investigators and their partners to other public and private quality improvement initiatives in their specific fields. With AHRQ's focus on portfolios, such support for grantees creates potential synergies across programs and connections between similarly focused grants that may be conducted under diverse auspices.  Although not all project officers have such skills, it may be valuable to encourage AHRQ staff to think more creatively about how best to use the knowledge and enthusiasm they bring to help leverage the work of individual grantees.

AHRQ also may benefit from a more careful review of findings in particular topic areas, with a view towards forming tighter connections with other AHRQ initiatives and efforts in the same areas. For example, the results of the three PFQ projects that focused on long-term care could be examined to determine how their results could be leveraged with current quality initiatives in the long-term care field. AHRQ staff with expertise in diabetes prevention and control could examine the results of the five PFQ projects that focused on control of diabetes in primary care or home health settings to assess opportunities for spreading effective approaches more broadly. They might also help the PIs of those projects connect with leaders in the diabetes prevention and control field.  The experience of the purchaser-led grants similarly provides important input on the factors that promote or inhibit purchasers from leveraging their influence to promote change in quality incentives and care within communities.

AHRQ could widely share the synthesis of findings and lessons in this evaluation, supplemented by final results from PFQ grantees that will be produced by September 2007.  Aside from this report and publications by individual grantees, there are currently no other ways to easily obtain information on what the PFQ program was and what grantees accomplished. The program remains relatively invisible, a belief shared by grantees and many of those directly associated with the program at AHRQ. AHRQ staff in various parts of the organizations should consider how best to translate the results of the most promising projects to relevant providers and professionals in the field.

3.  Appropriate Use of Quality Improvement Tools and Techniques for Translating Research into Practice

Several PFQ projects made important advances in testing and demonstrating the effectiveness of new tools and techniques for helping providers adopt or more fully implement clinical care guidelines. They include the effective use of appropriately scaled information technology, the development of practice-based CME, the integration of performance measures into electronic health records, and the design of quality reward and incentive programs by purchasers.  National and local quality improvement leaders wishing to replicate these strategies on a bigger scale can draw on the lessons of the PFQ projects. While some PFQ principal investigators have already begun to translate their success into lessons in these other fields, AHRQ staff can provide further support for these efforts.

To take one example, several PFQ projects made important advances in introducing information technology to health care facilities or to individual physician practices to aid in tracking adherence to clinical guidelines or to performance standards. The ISIS-sponsored PFQ project is a featured case study in the AHRQ National Resource Center on Health Information Technology20 and leveraged its success in the PFQ project to obtain new funding under AHRQ's Transforming Healthcare Quality through Information Technology (THQIT) Implementation Grants. Other PFQ projects have had some success as well, but efforts are needed to bring them to the attention of experts in the HIT field, so their lessons or implications for IT development in particular settings can be more carefully assessed.   Assessment of the PFQ project results might also be performed to determine if they should be included in AHRQ's new Innovations Clearinghouse. 

4. Future Design of Programs for Translating Research into Practice

PFQ provides valuable insights about the importance of agency leadership and program structure to the successful transition to new approaches to funding and translation work. From our review of the PFQ experience, we suggest several lessons important to the success of future programs seeking to translate research to practice. 

  • PFQ highlights the importance of senior leadership guidance on refining program strategy over time, not just when new programs are being conceived.
  • The selection and placement of program directors is important. AHRQ can do a better job of considering explicitly the structural constraints associated with the program director's role in order to pick and position individuals to increase their effectiveness in working with staff across the agency, and in communicating with top leadership. AHRQ's current structure makes the role of program director in a cross-center program like PFQ very challenging.  Since center directors are not held accountable for the program's success, responsibility rests with the program director to marshal the necessary resources, guide and motivate project officers to oversee grantees, and maintain commitment to the program's vision and goals through staff turnover and changing center and agency-wide priorities. The challenges are similar for project officers who get little support for actively supporting grantees in a cross-center program. Consideration might be given to adjusting staff workloads and incentive structures to reward staff for this type of grant oversight work.
  • Agency managers need to think through more clearly their expectations for cross-grantee work.  While some PIs valued the AHRQCoPs meetings and subcommittee work, the majority of them expressed frustration with the meetings, because they took away valuable funding, time and attention from their own projects and were not well-structured to foster synergy among the projects. AHRQCoPs and its subcommittees are producing a set of articles on partnership functions and lessons, to be published in a forthcoming special journal supplement. However, these activities and any learning they produced were linked only tangentially to the grantees' work and hence provided limited benefits to most of their projects. If AHRQ expects grantees in a program to work collaboratively, the final products should be more clearly defined and communicated to applicants in advance, and the agency should make clear to prospective grantees the amount of time and effort this activity will require.
  • AHRQ needs to better match grantee selection criteria to the goals of cross-grantee work. The PFQ grantees were too heterogeneous to foster significant collaboration, particularly without a strong content or focus that was relevant to all their needs. Programs like PFQ that seek to attract well-connected national and regional organizations whose base is outside of research also need to appreciate better the demands on the time of these individuals, which may mean limiting reporting requirements and collaboration work to the essential core.

In sum, PFQ generated capacity and knowledge that can support broader AHRQ's efforts to translate research into practice.  Harvesting its potential will further leverage AHRQ's $20 million investment in PFQ and enhance the strategic value of this program as an early pioneer whose experience and lessons can inform attempts to translate research to practice on a broad scale. 


19. Partnerships for Quality. Fact Sheet. AHRQ Publication No. 04-P004, March 2004. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/qual/partqual.htm

20. "Long-Term Care Facilities Embrace Health Information Technology," located on the AHRQ HIT National Resource Center Web site: http://healthit.ahrq.gov; select AHRQ-Funded Projects, and then on Case Studies.


Return to Contents
Proceed to Next Section

 

The information on this page is archived and provided for reference purposes only.

 

AHRQ Advancing Excellence in Health Care