Skip Navigation Archive: U.S. Department of Health and Human Services
Archive: Agency for Healthcare Research Quality
Archive print banner
AHRQ Performance Plans for FY 2002 and 2003 and Performance Report for FY 2001

This information is for reference purposes only. It was current when produced and may now be outdated. Archive material is no longer maintained, and some links may not work. Persons with disabilities having difficulty accessing this information should contact us at: Let us know the nature of the problem, the Web address of what you want, and your contact information.

Please go to for current information.

FY 2003 Outcome Measures for Measuring Improvement in the Quality of Healthcare

Key Outcomes—Improving the Quality of Health Care, FY '03
  • Produce the First Annual National Quality Report
  • Develop a CAHPS® module for consumer assessments of health and services received in nursing homes
  • Develop CAHPS® assessment instruments for people with mobility impairments

Program Outputs Needed to Achieve Expected Results

At the budget level requested in the FY 2003 President's Budget, AHRQ will be able to maintain grant and contract commitments which address translating research into practice, MEPS, HCUP, CAHPS and patient safety. Reductions will be made to grants and contracts in other areas.

  • Maintain the number of projects being funded that bring healthcare quality information to the public in an understandable, user-friendly manner that facilitates its use in decision-making. Baseline: FY 2001
  • Identify private sector data to be used in future National Quality Reports
  • Identify a core set of quality measures to be used to report on progress in improving the overall quality of health care.

Indicators AHRQ Will Use to Measure Success in this Area Over the Next Two Fiscal Years

  • Track improvements in the quality of care as measured by a core set of measures that have been proven to be reliable and valid.
  • Track how the public utilizes quality measures

2.1.2.B Program Description and Context: Measuring Improvements in Patient Safety

Patient safety is a top priority in the nation today. It is estimated that up to 98,000 Americans die each year as a result of preventable medial errors (Kohn 1999). The majority of these errors are a result of systemic problems rather than poor performance by individual providers. Although the Untied States provides some of the best health care in the world, there are a significant number of patients that are being harmed as a result of the process of health care.

In Fiscal year 2001, AHRQ re-named the Center for Quality Measurement and Improvement (CQMI) the Center for Quality Improvement and Patient Safety (CQuiPS). This was tangible evidence of AHRQ's efforts to re-focus activities to improve the quality of health care Americans receive, and address preventable patient errors by reducing the risk of patients being harmed by the process of health care.

The goal of patient safety is to reduce the risk of injury and harm from preventable medical errors. This goal can be accomplished by the removal or minimizing of hazards, which increase the risk of injury to patients. Three steps must be followed to ensure that the "epidemic" of medical errors is contained. These steps are:

  • Identify the causes of preventable errors and the hazard that increase the risk of injury to patients
  • Implement patient safety practices that eliminate known hazards and reduce the risk of injury to patients and create a positive safety culture.
  • Maintain Vigilance to ensure that a safe environment continues and a positive safety cultures are maintained.

Key phrases are risk, adverse events, no-harm events, near miss, and detection.

Risk is the possibility/probability of occurrence or recurrence of an event multiplied by the severity of the event. Hazard is anything that can cause harm (NHS 2000). An event is a deviation in an activity or technology, which leads towards an unwanted, negative consequence. (Freitag and Hale 1997). Events can be classified in three different categories (Battles 2001).

An adverse event/misadventure is an occurrence during clinical care which results in physical or psychological injury or harm to a patient or harm to the mission of the organization. A no-harm event is an event that has occurred but resulted in no actual harm although the potential for harm may have been present. Lack of harm may be due to the robust nature of human physiology or pure luck. An example of such a no-harm event would be the issuing of an ABO incompatible unit of blood for a patient, but the unit was not transfused and was returned to the blood bank.

A near miss as defined by Van der Schaaf (1991) is an event in which the unwanted consequences were prevented because there was a recovery by identification and correction of the failure, either planned or unplanned. Heinreich (1941) developed the iceberg concept of accidents and errors. The part of the iceberg above the water represents events that cause major harm; below the water are no-harm events as well as near misses.

Strategies for Improving Patient Safety

The overall goal of reducing medical errors and improving quality and patient safety is an overall HHS goal, which is shared among AHRQ, Food and Drug Administration (FDA), Center for Medicare and Medicaid Services (CMS), HRSA and Indian Health Service (IHS). AHRQ's research contribution focuses on creating basic knowledge and evaluating the strength of existing evidence which provides information that can be turned into actions by those who make clinical decisions, purchasers and providers who make decisions about what services to use, pay for and how to structure those services, as well as by public policy makers.

In cooperation with other HHS partners AHRQ will seek to improve the quality healthcare and reduce medical errors by 1) accelerating the implementation of existing quality measures and safety practices 2) developing capacity and new practices for Quality and Safety Improvements and 3) detecting safety hazards and monitoring improvements in healthcare Safety and Quality.

The Agency believes the best way to approach improving patient safety is through an integrated set of activities to design and test best practices for reducing errors in multiple settings of care; develop the science base to inform these efforts, as well as to improve provider training in the reduction of errors; capitalize on the advances in information technology to translate proven effective strategies into widespread practice; and build the capacity to further reduce errors in the future. AHRQ's goal of improving the quality and safety of the healthcare that Americans receive cannot be achieved in a single year, but must follow a systematic progression of activities over a number of years. In addition, AHRQ's activities should not be viewed in isolation, but should be combined with similar efforts in other HHS departments, to produce the desired improvements in quality and safety.

In FY 2003, activities will build on the progress made in FY '01 and '02 and activities directed at stage two of the medical error epidemic will be added to the portfolio, with a primary focus on minimizing the risk of preventable injuries.

Detecting safety hazards and monitoring improvement in Safety and Quality

Detection is the first step in error management. From an organizational point of view it is important that error detection rate be high because errors that are not detected can have disastrous consequences (Zapt & Reason 1994). Thus one of the goals of error management is to increase detection and reporting rates to decrease risk of harm to patients. Detection Sensitivity Level (DSL) is the number of events reported to an organization. High reporting rates indicate a high DSL while few events reported indicates a low DSL. To achieve a high DSL an organization must eliminate impediments to reporting. Event Severity Level (ESL) is the level risk of the events reported. Over time the event severity level should go down as an indicator of successful error management while the detection sensitivity level (DSL) remains high (Kaplan 1998). DSL represents information while ESL represents risk.


This image displays that Detection Sensitivity Level (DSL), which is the number of events reported to an organization, should have high rates. To achieve a high DSL an organization must eliminate impediments to reporting. Event Severity Level (ESL), which is the level risk of the events reported, should have low rates.  Over time the ESL should go down as an indicator of successful error management while the DSL remains high (Kaplan 1998).  DSL represents information while ESL represents risk.


Developing a system which will consistently and reliably identify potential hazards to patient safety will require developing a national system of patient safety reporting and monitoring. For example, existing data collection mechanisms, if properly coordinated and standardized, could provide a powerful national asset as an early warning system for patient safety hazards, providing pooled data for analysis to develop solutions to patient safety problems, and for tracking progress in patient safety.

In addition, a successful monitoring system will require a direct investment in IT infrastructure and enhancements to improve safety monitoring. Healthcare is well behind other industries in harnessing the potential of information technology. Developing the healthcare IT infrastructure is key to monitoring the improvements in patient safety. The development of options and information to support investments in IT infrastructure, including cooperative agreements to encourage IT investment and develop national models of IT infrastructure.

Finally, developing data standards and vocabulary to ensure that patient safety information can be pooled, analyzed, and used to identify hazards and safety practices will be essential. The lack of standardized coding and language could result in a healthcare information system "Tower of Babel". This initiative, in conjunction with the Department and sister agencies, will strive toward the development of patient safety information standards, development of voluntary consensus on those standards, and incorporation of those standards into existing Departmental databases.

Accelerate the implementation of existing patient safety practices

In July 2001, the Agency released a report, entitled Making Health Care Safer: A Critical Analysis of Patient Safety Practices, which examined the evidence supporting 79 promising safety interventions. Many of these proven lifesaving interventions, such as the use of medications to reduce the risk of heart attacks and infections in surgical patients, are not consistently implemented.

Key to developing and implementing this strategy will be implementing local safety improvement priorities and improving the information available to the public on patient safety. Healthcare organizations and systems must be challenged to implement proven patient safety practices. Participating organizations and systems should be encouraged develop their own practice implementation practices from those identified in the Agencies FY 2001 report and will be provided funding to facilitate the system's investment in implementing those practices. Finally, these organizations and systems will be required to generate information on the barriers faced in implementing the practices as well as measure the impact of the intervention on patient safety. Cooperative agreements will include a provision to make continued funding of the program contingent upon a demonstrated ability to implement the identified priorities and measure their impact.

In addition, to order to improve the information available to the public on patient safety the Agency will use its Consumer Assessment of Health Plans Survey (CAHPS) program, which is responsible for the development of state of the art survey and reporting instruments, to provide information about patient's experience of care; to improve the relevance of systems based information on patient safety for the public; support value-based purchasing initiatives; and develop mechanisms for public reporting on evidence-based safety interventions.

Developing Capacity and New Practices for Safety Improvement

The Agency's July 2001 report on patient safety interventions identified a number of promising interventions for which more effectiveness evidence is required prior to general implementation. Many potential patient safety practices drawn primarily from non-medical fields, such as the use of simulators, bar coding, "swipe" technology, and crew resource management, require additional study to clarify their value in the health care environment. The success of this strategy will depend on the continuing development and evaluation of new safety interventions, improving local patient safety infrastructure and capacity, and developing reliable information for purchasers and the public to support wise decision-making.

Return to Contents

FY 2003 Outcome Measures for Measuring Improvement in Health Care Safety

AHRQ is committed to identifying ways of continually improving patient safety in an environment of high quality health care. Its investments sponsor research that helps to measure the overall safety culture within health care institutions. The agency's performance goals and measures will be used to gauge progress as advances are made toward increasing event reporting and reducing risk of harm.

Key Outcomes—Improving the Safety of Health Care, FY '03

  • Nationally, 12 health facilities or regional initiatives will be funded to implement interventions and service models on patient safety improvements by 2004.
  • On-site patient safety experts in 10 states and technical assistance to improve patient safety by 2004

Program Outputs Needed to Achieve Expected Results

At the budget level proposed in the President's Budget AHRQ will maintain funding for our Translating Research into Practice grants and contracts. At this level expiring grants are not continued and the dollar amount reduction will be spread across non-patient safety and training grants.

  • Invest $3 million in new activities designed implement local safety improvements priorities by providing incentives to put systems-based interventions in place in healthcare organizations.
  • Invest $2 million in new programs designed to provide technical assistance to states and provide on-site patient safety experts to improve local capacity.
  • Produce at least one synthesis of research findings and practical applications of AHRQ's research which will be available to clinical decision makers, health systems and policy makers
  • Report the interim data on the extent the patient safety best practices identified in the July 2001 EPC report have been adopted by health care institution
  • Identify the number and types of adverse events, no-harm events, and near miss events reported in demonstration projects (Baseline established)

Indicators AHRQ Will Use to Measure Success in this Area

  • Track the number and types of adverse events, no-harm events, and near miss events reported.
  • Track the number of patient safety interventions being put into practice.

External Factors influencing success: AHRQ's ability to meet its target goals is dependent upon complementary programs being implemented in CMS and OASPE.

2.1.3 Performance Goal 3: To develop the evidence base for policy makers and health systems to use in making decisions about what services to pay for, how to structure those services, and how those services are used and accessed.

Program Description and Context Develop Strategies for Making Decisions

Adequate access to health care services continues to be a challenge for many Americans. This is particularly true for the poor, the uninsured, and members of minority groups, rural residents, and other priority populations. In addition, the changing organization and financing of care has raised new questions about access to a range of health services, including emergency and specialty care. At the same time, examples of inappropriate care, including over utilization and misuse of services, continue to be documented. Through ongoing development of nationally representative and more specialized databases, the production of public-use data products, and research and analyses conducted by AHRQ staff and outside researchers, the Agency addresses critical policy issues pertaining to the access, cost and use of health care.

Research related to the effects of payment and organizational structures and processes on the cost, quality and equity of health care services are essential components of this research initiative. Research results are intended to 1) improve clinical practice, 2) improve the health care system's ability to provide access to and deliver high quality, high-value health care, and 3) provide policymakers with the ability to assess the impact of payment and organizational changes on outcomes, quality, access, cost, and use of health care services.

Responding to the Institute of Medicine's (IOM) report, "Crossing the Quality Chasm," AHRQ will fund applications that provide rigorous, objective, and essential evidence required by public and private decision-makers seeking to understand and improve the health care system, to make changes in health care delivery, insurance, and financing, and to manage the system in a manner that would induce efficient, effective, equitable, accessible and timely health care.

Important issues to be addressed by such research include: 1) How do different payment methodologies and financial incentives within the health care system affect health care quality, costs, and access? a) How do payment methodologies affect the behavior of health care organizations and individual providers? b) Which payment arrangements among patients, providers, and health plans enhance patient-centered knowledge of and involvement with treatment regimens? c) How do payment policies affect decisions about the purchase and selection of health services and health insurance? What is the role of quality in such decisions? What are the effects of such decisions on health care costs? 2) What has been the impact of purchaser and public sector initiatives on quality, costs, and access to health care and health insurance? Of particular interest would be the impact of employer and coalition efforts on the quality and cost-effectiveness of care in the marketplace, the impact of State efforts to monitor and improve access and quality, and the impact of public and private payment changes on access to health care and to health insurance for vulnerable populations. 3) What organizational structures and processes are most likely to sustain high-quality, efficient, effective, timely, and accessible health care? 4) How do different patterns and levels of market competition affect the quality and cost of care?

In addition, AHRQ will fund projects in basic methodological work to support such research, including: development of payment methodologies; improvements in analytical and empirical methods required to simultaneously address issues of efficiency, quality, and equity; and improvement in data collection methods and qualitative methods needed to understand the structure of new health care organizations and an evolving health care system.

Strategies to Improve Access, Foster Appropriate Use and Reduce Unnecessary Expenditures

Excellence Centers to Eliminate Ethnic/Racial Disparities (EXCEED)

The health of Americans has improved over the past few decades, but not all Americans have benefited equally. Many ethnic and racial groups have not shared in the advances in health outcomes and health care. To understand the causes and factors of these inequalities, in September 2000, AHRQ awarded grants to nine Centers to conduct a series of related studies. Now known as "Excellence Centers to Eliminate Ethnic/Racial Disparities" (EXCEED), the grants bring together teams of both new and experienced investigators in a 5-year effort to analyze underlying causes and contributing factors for racial and ethnic disparities in health care and to identify and implement strategies for reducing and eliminating them. The Centers of Excellence approach enables assembly of a critical mass of investigators to address a group of projects linked by a central theme, such as communication or cultural competency. In addition, the Centers are able to train new investigators with an interest in minority health services research.

The EXCEED program joins a series of AHRQ initiatives aimed at improving the health and health care of priority populations and eliminating racial and ethnic disparities in health outcomes and in health care access and service delivery. AHRQ expects that lessons learned from understanding and eliminating racial and ethnic disparities in health and health care as well as the practical tools and strategies to eliminate these disparities would be generalizable beyond the communities studied. By focusing on the attributes of the ethnic and racial groups, the underlying etiologies for the disparities, and components and conditions of interventions to eliminate the disparities, these projects should produce findings that are widely applicable for minority as well as majority populations across the county.

Integrated Delivery System Research Networks (IDSRNs)

The Integrated Delivery System Research Networks (IDSRN) is a new model of field-based research that links the Nations top researchers with some of the largest health care systems to conduct research on cutting edge issues in health care on an accelerated timetable.

Designed by AHRQ, the IDSRN was developed explicitly to capitalize on the research capacity of, and research opportunities occurring within, integrated delivery systems. The network creates, supports, and disseminates scientific evidence about what works, and what does not work in terms of data and measurement systems and organizational "best practices" related to care delivery and research diffusion. It also provides a cadre of delivery-affiliated researchers and sites to test ways to adapt and apply existing knowledge.

As a group, the IDSRN provides health services in a wide variety of organizational care settings to over 34 million Americans, including the privately insured, Medicare and Medicaid patients, the uninsured, ethnic and racial minorities, and rural and inner-city residents. Moreover, each of the nine IDSRN partners have the following three unique attributes that make it particularly well suited for conducting time-sensitive research:

  • Available Data—each of the IDSRN partners collect and maintain administrative, claims, encounter and other data on large populations that are clinically, demographically and geographically diverse
  • Research Experience—IDSRN partners include some of the country's leading health services researchers, with expertise in quantitative and qualitative methodologies as well as first hand knowledge of emerging issue areas which have critical policy or managerial implications
  • Management authority to implement health care interventions—IDSRN partners have responsibility for managing delivery systems and are in a position to implement financial and organizations strategies (e.g., selective contracting with physicians and hospitals for specific approaches to diffuse clinical guidelines within the system) with an evaluation component.

Primary Care Based Research Networks (PBRNs)

Over the past decade, primary care practice-based research networks (PBRNs) have emerged as a promising approach to the scientific study of primary care. A PBRN is a group of ambulatory practices devoted principally to the primary care of patients, affiliated with each other (and often with an academic or professional organization) in order to investigate questions related to community-based practice. Typically, PBRNs draw on the experience and insight of practicing clinicians to identify and frame research questions whose answers can improve the practice of primary care. By linking these questions with rigorous research methods, the PBRN can produce research findings that are immediately relevant to the clinician and, in theory, more easily assimilated into everyday practice. In addition, data is readily available which will provide useful information on who receives care, where and how often.

In September 2001, 18 of these networks were awarded continuation grants (cooperative agreements) to conduct network-defining surveys, using standardized instruments. The surveys will provide baseline data on the clinicians enrolled in each network, the services provided, and the characteristics of patients receiving those services. Potential uses of the data include practice benchmarking and guiding the selection and design of specific PBRN research projects. Four of the networks were awarded additional funds to pilot test and evaluate electronic methods of collecting and aggregating practice-derived research data. Two networks were awarded additional funds to assess clinician and patient knowledge and attitudes about protecting the privacy and confidentiality of research data.

AHRQ's overall goal is to improve the capacity of PBRNs to expand the primary care knowledge base and to establish mechanisms that will assure that new knowledge is incorporated into actual practice and its impact is assessed. Additional information on PBRNs is available at:

HIV Research Network

As HIV/AIDS spreads into different communities and as new therapies become available, policymakers require reliable information about the type and costs of the health care services that persons with HIV disease are receiving, so that informed resource-allocation decisions can be made. The HIV Cost and Services Utilization Study (HCSUS) is the first major research effort to collect information on a nationally representative sample of people in care for HIV infection. HCSUS is examining costs of care, utilization of a wide array of services, access to care, quality of care, quality of life, unmet needs for medical and nonmedical services, social support, satisfaction with medical care, and knowledge of HIV therapies.

HCSUS is composed of a core study and several supplemental studies. The core study has enrolled a national probability sample of 2,864 HIV-infected adults who were receiving ongoing or regular medical care in the first 2 months of 1996. Respondents were sampled from 28 urban areas and 24 clusters of rural counties in the continental United States. Patients receiving services in hospitals, clinics, and private practice settings were enrolled. HCSUS oversampled women and members of staff model health maintenance organizations, to obtain more precise information on these specific populations. Supplemental studies are examining HIV care delivery in rural areas, prevalence of mental and substance abuse disorders, oral health of HIV-positive individuals, and issues related to HIV-infected persons over 50 years of age.

HCSUS is funded through a cooperative agreement between the Agency for Health Care Policy Research (AHCPR) and RAND-a private nonprofit research institution in Santa Monica, CA. Supplemental funding has been provided by the Health Resources and Services Administration (HRSA), the National Institute of Mental Health (NIMH), the National Institute of Aging (NIA), the National Institute of Drug Abuse (NIDA), the National Institute of Dental and Craniofacial Research (NIDCR) (NIDR), and the Office of Minority Health Research (OMR). A consortium of private and Government institutions, centered at RAND, are conducting HCSUS. Local and national advisory groups have been established to facilitate communication between the HIV community and the research consortium.

In Fiscal Year 1999 and 2000, AHRQ and sister agencies (HRSA, SAMHSA, and the Office of AIDS Research at NIH) provided support to develop the first phase of the HIV Resource Utilization Data Coordinating Center (DCC). In Fiscal Year 2001, based on the promising results of the first phase, support from these agencies has continued, and plans have been made to expand the scope of this project.

The goal of this project is to provide timely and relevant information to policymakers about the resources utilized to treat persons with HIV disease. It is important to obtain timely data because quickly occurring therapeutic advances, the shifting epidemiology of HIV infection, and the rapid diffusion of managed care are having a major effect on patterns of health care resource utilization for HIV disease. More than $7 billion spent each year by Medicaid, Medicare, the Department of Veterans Affairs, and the Ryan White CARE Act to treat people with HIV disease. Information is needed by Federal and State policymakers to ensure that all persons with HIV disease have access to appropriate health care services.

The mechanism developed to address these issues is a multi-site HIV provider network, which is linked by a centralized data-coordinating center. The Data Coordinating Center (DCC) receives data from a large number of HIV providers around the nation in order to provide valid and reliable information about the determinants of resource utilization by persons with HIV disease. The Data Coordinating Center has the capability to receive, consolidate, analyze, and disseminate information about the care provided to persons with HIV disease, eventually operating in a continuous data receipt mode.

Healthcare Cost and Utilization Project (HCUP)

The unprecedented volume and pace of change in the U.S. health care system, and the fact that changes are not occurring uniformly across the country, require a new information paradigm in which scientifically sound, standardized databases and tools for using them are made available at the national, regional, and State levels.

The Healthcare Cost and Utilization Project (HCUP) is a Federal-State-industry partnership to build a standardized, multi-State health data system. HCUP is maintained by the Agency for Healthcare Research and Quality (AHRQ, formerly the Agency for Health Care Policy and Research). AHRQ has taken the lead in developing HCUP databases, Web-based products, and software tools and making them publicly available.

HCUP comprises a family of administrative longitudinal databases—including State-specific hospital-discharge databases and a national sample of discharges from community hospitals—and powerful, user-friendly software that can be used with both HCUP data and with other administrative databases.

HCUP databases fill a unique niche and are increasingly being tapped by analysts and researchers interested in hospital utilization, access, charges, quality, and outcomes. The data are used to:

  • Describe patterns of care for uncommon as well as common diseases.
  • Analyze hospital procedures.
  • Study the care of population subgroups such as minorities, children, women, and the uninsured.

Researchers and policymakers use HCUP data to identify, track, analyze, and compare trends at the national, regional, and State levels.

Return to Contents
Proceed to Next Section


The information on this page is archived and provided for reference purposes only.


AHRQ Advancing Excellence in Health Care