This information is for reference purposes only. It was current when produced and may now be outdated. Archive material is no longer maintained, and some links may not work. Persons with disabilities having difficulty accessing this information should contact us at: https://info.ahrq.gov. Let us know the nature of the problem, the Web address of what you want, and your contact information.
Please go to www.ahrq.gov for current information.
4th Indicator: Use of evidence reports or technology assessments and access to NGC site informed organizational decisionmaking in at least 4 cases and resulted in changes in health care procedures or health outcomes.
Results: The impact of AHRQ-sponsored research is evident in the following examples of organizational decision making informed by evidence reports, technology assessments and data availability at the NGC Web site:
- US Preventive Services Task Force (USPSTF)—USPSTF recommendations, based on AHRQ-sponsored systematic evidence reviews (SER's), formed the basis of an effort by the nonprofit group Partnership for Prevention to prioritize recommended clinical preventive services to facilitate decision making among purchasers, payers, and health plans about coverage and delivery of clinical preventive services. The study, released in June 2001, prioritizes clinical preventive services based on their health impact and cost effectiveness. The release of this report received significant national media attention and the National Healthcare Purchasing Institute will distribute 4,000 copies of a related monograph for employer-purchasers. It is expected that this report will have widespread impact on policy makers and organizational decision makers in considering issues related to coverage of clinical preventive services.
- Rehabilitation for Traumatic Brain Injury in Children and Adolescents—Based on results of the AHRQ-supported evidence report on this topic, Aetna U.S. Healthcare announced a change in national coverage policy. As of February 2, 2000, consistent with the findings of the evidence report, Aetna allows payment for cognitive rehabilitation as adjunctive treatment of cognitive deficits when specified criteria are met.
- Cervical Cytology—The findings of this evidence report have been considered by two large HMOs—Excellus Healthcare (Blue Cross/Blue Shield of upstate NY) and Henry Ford (Detroit)—to develop coverage policies with respect to cervical cancer screening using the new liquid-based technologies. In addition, the cost-effectiveness model from this report is being used by a major pharmaceutical company to assist in the planning and evaluation of a human papillomavirus vaccine.
- Management of Uterine Fiborids—This evidence report is being used by the Society for Cardiovascular and Interventional Radiologists to design a prospective registry for measuring outcomes in women undergoing uterine artery embolization. Study design and choice of outcome measures were largely guided by the findings of the evidence report. Intermountain Health Care, a large integrated health system in Utah, is considering use of the report to develop guidelines for management of women with fibroids, as well as potential evaluations of the impact of the guidelines on patient care.
- Criteria for Clinical Guidelines—An article based on this AHRQ-sponsored EPC report that reviews AHCPR's former guidelines and provides criteria for when clinical guidelines should be updated an/or withdrawn, will be published in the British Medical Journal and will be useful to organizations that develop and adapt clinical practice guidelines in making decisions about updating and withdrawing outdated guidelines.
5th Indicator: At least 3 examples of how research informed changes in policies
or practices in other Federal agencies.
Results: AHRQ research has been used by several federal agencies to structure policies and practices. Examples are listed below:
- AHRQ's research portfolio on low income populations has informed HRSA's work on evaluating and improving the health care safety net.
- AHRQ-funded research on guideline development and implementation has influenced the recommendations on professional education/performance measurement made by the Secretary's Advisory Committee on Genetic Testing (an NIH-led effort).
- AHRQ-sponsored research conducted by Richard Zimmerman, M.D. on the effect of the Free Vaccine program on delivery of immunizations in primary care settings has supported the efforts of CDC to continue that program.
- The National Cancer Policy Board issued a white paper on the volume-outcome relationship in cancer surgery on July 12 that includes a table based on HCUP analyses. The paper contains research recommendations that were informed by the clarity HCUP data brought to the understanding of potential impact of selective referral recommendations.
In addition, AHRQ is working to document impact case studies of how research has facilitated changes in policies or practices in other federal agencies. AHRQ-sponsored workshops, presentations, and technical assistance that address use of AHRQ tools generate a great deal of interest in products resulting from AHRQ investments. The following exemplify this:
- AHRQ conducted a 1.5-hour methods workshop on HCUP data at the AHSRHP conference. Despite being held as one of the last sessions, there were over 55 participants and there was heightened interest in the data.
- At another session AHSRHP, AHRQ's staff did a presentation on the use of HCUP data and the Quality Indicators for the National Quality Report.
- In addition, staff participated in two sessions at the Child Health Services Research conference, doing a presentation on the use of HCUP data and tools for research on children's hospitalizations, and moderating a session on state data for child health services research.
AHRQ-supported research was the information source for several government agencies. The following demonstrate how diverse these agencies are:
- CBO—received information on trends in mental health hospitalizations from 1990-1997. These data are useful to work they are doing on parity in coverage of mental health services (based on NIS data).
- The National Quality Forum—received information on the most common reasons for hospitalization by race for 7 HCUP SID states.
- ASPE—received information on quality of care for Medicare patients.
6th Indicator: Achievable Benchmarks of Care are used for quality improvement
activities by Peer Review Organizations.
Results: A new CAHPS RFA was released in May 2001 that will include the development of a module for use in quality improvement activities.
7th Indicator: Use of dental performance measures by dental service and insurance organizations.
Results: Draft instruments for Nursing Home CAHPS, Dental CAHPS, ECHO are currently undergoing testing. In addition, AHRQ has obtained expert input into the development of a CAHPs module for persons with mobility impairments.
8th Indicator: HCUP quality indicators incorporated into efforts by hospital
associations and hospitals to improve the quality of care.
Results: Quality Indicators from AHRQ-funded research, used to assess health care quality, are increasingly an integral part of providers' efforts toward improving the level health care quality in the nation. For example, the Healthcare Association of New York State (HANYS) provides feedback to hospitals using the HCUP Quality Indicators and has described a number of examples of how hospitals have changed internal practices or have entered into collaborations with surrounding communities to improve the quality of care:
- One New York hospital reported that a hospital-wide intervention to prevent venous thrombosis was initiated in response to higher than average rates of post-surgical venous thrombosis reported through the HCUP QIs.
- A group of hospitals worked with primary care providers in the community to improve the outpatient treatment of diabetes because the rates of admission for diabetes complications measured using the HCUP QIs were higher than for other communities.
- Another group of hospitals worked with the state chapter of the College of Obstetricians and Gynecologists to reduce high rates of low birth weight identified using the HCUP QIs by fostering appropriate prenatal care in the surrounding community.
National Guideline Clearinghouse
9th Indicator: At least 10 users of the National Guideline Clearinghouse will use
NGC site to inform clinical care decisions.
Results: The following are examples from the June 2001 NGC Second Annual Customer Satisfaction Survey of physician comments about how they use the NGC to inform clinical care:
- I can find appropriate choice of therapies for the patient I am taking care of.
- It is help in my clinical practice.
- This is a way to keep updated.
- I use it to search for guidelines for treatment and diagnosis to formulate the best approach for [my patients].
- Advice for decision making in daily routine.
- An excellent scientific source for my practice.
- Answering clinical questions.
- Apply evidence-based medicine in clinical practice.
- As a handy source for clinical question resolution.
- [I use the NGC] as a pediatrician doing patient care and an administrator developing a system for EBM at point of care.
- I am an academic Physician (Endocrinologist) and I need a rapid access to all kinds of Medical information pertinent to Internal Medicine and Endo/Metaboloism. I recently discovered the NGC site, which, I believe, is the most "cost/effective" site available in Internet for a rapid, comprehensive review of the state of art about Medical Guide.
10th Indicator: Guideline development or quality improvement efforts by users will
be facilitated through use of NGC in at least 5 cases.
Results: The following are examples from the June 2001 NGC Second Annual Customer Satisfaction Survey of physician comments about how they use the NGC for guideline development or quality improvement efforts:
- As a critical care anesthesiologist in charge of a department I found it very helpful to base our protocols on solid ground.
- As Chief of the Medical Staff at a military hospital, I use the guidelines to help standardize rational approaches to health care and treatment for the physicians that work in my facility.
- Assistance in developing new local guidelines
- Development of guidelines for a pediatric care network
- Develop a strategic plan for my hospital re methods of standardization of clinical care
- Developing clinical guidelines for my group practice
- Developing critical paths in my hospital
- Developing guidelines for a managed care organizations
- Developing guidelines for our community health center's use
- Developing physician quality assessment based on compliance with guidelines
11th Indicator: NGC information will be used to inform health policy decisions in at least 2 cases.
Results: The following are examples from the June 2001 NGC Second Annual Customer Satisfaction Survey of Hospital/Health Administrator, Purchaser/Employer, and Policymaker comments about how NGC has been used to inform health policy decision:
- As a resource for policy formation at my health care facility.
- In the development of medical policies and clinical guidelines.
- Review of national guidelines for medical policy development.
- Coverage decision, clinical policy and guideline development.
- Reviewing it for reference in our Health Alaskans 2010 planning documents.
12th Indicator: Improvements in clinical care will result from utilization of NGC
information in at least 3 cases.
Results: The following physician comments about improvements in clinical care resulting from use of the NGC are excerpted from the June 2001 NGC Second Annual Customer Satisfaction Survey:
- [I use the NGC because] I want to improve my clinical practice level
- [I use it to search] for guidelines for treatment and diagnosis to formulate the best approach for [my patients]
- As Chief of the Medical Staff at a military hospital, I use the guidelines to help standardize rational approaches to health care and treatment for the physicians that work in my facility.
13th Indicator: Two thirds of former pre- and postdoctoral institutional award
trainees are active in the conduct or administration of health services research.
Results: A comprehensive evaluation of career paths and productivity of former pre- and postdoctoral trainees supported on institutional training grants was undertaken. Data indicate that former trainees are employed and actively involved in the conduct or administration of health services research. Results have demonstrated that 2/3 of pre-doctoral students are actively engaged in health services research activities (the majority of the remaining 1/3 have not yet completed their training). Over 90% of postdoctoral trainees supported by AHRQ are employed in research-related positions. The majority of former trainees are primarily in academia, though with an increased number opting for careers in nonacademic settings. Overall, three quarters continue to publish in academic journals since completion of their training, 1/3 have published books or chapters in books, and approximately 1/4 have published technical reports. Over 50% were investigators on grants or contracts related to health services research projects, a rate which escalates to 2/3 among former postdoctoral trainees.
In the spirit of continuous quality improvement, AHRQ also queried former trainees on their training experiences. Satisfaction rates were captured across multiple dimensions ranging from program content, to mentoring, to career relevancy. All were extremely high (over 90% were satisfied). Feedback provided by former students served as a basis for the development and refinement of career development programs sponsored by AHRQ. A conference on curriculum development was convened, at which a variety of educational institutions exchanged approaches to and innovations in preparing pre as well as postdoctoral students for careers in health services research. Attention was paid to the formation of public/private partnerships, responsiveness to community interests, and training in cultural competency. In addition, AHRQ embarked on programs designed to foster the research career development of newly minted researchers in order to prepare them to compete successfully with established researchers for support. Effort was also placed on building and extending a community of health services researchers across current training programs, as well as in states which do not traditionally receive significant HSR funding and institutions which traditionally serve minority students.
14th Indicator: Fund the establishment of a patient safety program evaluation center that shall develop and implement an overall evaluation plan for patent safety projects funded by AHRQ.
Results: A task order has been issued to establish an implementation plan for a patient safety program evaluation center. Proposals were due by 8/30/2001 and are currently under review.
Objective 4.2: Evaluate the impact of MEPS data and associated products on policymaking and research products.
1st Indicator: Use of MEPS data in AHRQ research applications will increase by
10 percent over number received in baseline period of 2000.
Results: The number of grant applications submitted to AHRQ that proposed the use
Of MEPS data decreased from 4.7% in 2000, to 2.5% in 2001. Out of a total of 715 grant applications submitted to AHRQ in 2001, 18 required the use of MEPS data. 3 of these applications were funded. This decrease may have been due to AHRQ targeting the funding of grants in 2001 on research areas that are not supported by MEPS data, such as patient and worker safety. To improve access to technical assistance CCFS conducted 12 user workshops and answered 920 technical assistance questions. Approximately 100 users were added to the MEPS LISTSERV® increasing the subscriber number to 400. CCFS staff also staffed the AHRQ booth at professional meetings to answer MEPS related questions.
2nd Indicator: Feedback from MEPS workshop participants indicating that the
workshops were useful and timely.
Results: In 2001 CCFS conducted 12 MEPS user workshops. 95 percent of participants completing workshop evaluations said that the workshop met their primary goal for attending the workshop.
3rd Indicator: At least 5 examples of how research using MEPS has been used to inform decisions by Federal, state and private sector policymakers.
Results: In 2001 CCFS published 47 journal articles in peer reviewed journals, 3 Chart Books, 1 Methodology Report, 2 Research Findings Reports, and 1 Highlight publication. (see attachments). All of these publications provide valuable input into health care policy formulation. In 2001 CCFS staff also made more than 100 MEPS related presentations at professional meetings or for heath related organizations. The following examples highlight the use of MEPS data for policy decision:
- The staff of Representative Istook (Oklahoma) requested a tabulation of the top health conditions ranked according to condition specific health expenditures.
- In June, during the Congressional debates on the patients Bill of Rights, AHRQ received several inquiries about MEPS-IC health insurance data, including requests from Senator Kennedy's office and from the Heritage Foundations. A follow-up call to Senator Kennedy's office confirmed that they found the information on our web site to be informative and useful. According to Ralph Rector of the Heritage Foundation "When it comes to health insurance premium information, the MEPS-IC is the only game in town."
- Two States, Wisconsin and Arkansas purchased additional sample in the MEPS-IC this year to improve the MEPS-IC estimates for their states. This is related to work we have been doing for 20 States that are studying ways to expand health insurance coverage for low income children, workers, and families by coordinating public subsidies with private (employment-based) coverage, both under SCHIP and potentially more broadly. Over the past 6 months, we have provided special data tabulations to representatives from these States that provides additional information from the MEPS-IC survey to help them understand the structure of employment-based coverage in their States.
- AHRQ staffers prepared a literature review entitled "Does Spending on Prescription Drugs Reduce Spending on Other Health Care Services? A Preliminary Literature Review." This literature review, which was used as the basis for briefing staff in the Office of the Secretary, HHS, in April 2001. Featured among the papers reviewed was a recent study on this topic using MEPS data by researchers at Columbia University.
- AHRQ responded to a request from the Council of Economic Advisors for information on non-group premiums from the 1996 MEPS. The Council was provided with estimates by age group and single and family policies for the non-elderly population. The Council was also interested in more information on the type of policies purchased in the nongroup market, specifically whether these were high deductible plans. For those policies with linked MEPS HIPA data, AHRQ was able to show that many of the nongroup plans purchased in 1996 were high deductible policies. AHRQ also provided 1996 MEPS HC-IC estimates of private, non-group health insurance premiums by age group in response to requests from the Treasury, CBO, ASPE (to be included in a memo to the Assistant Secretary for Policy and Evaluation).
- AHRQ staff presented a paper on children's insurance at the National Academy of Sciences' "Workshop on Enhancing Methods for the State Children's Health Insurance Program" in June. The paper is meant to help states understand issues underlying the design of public health insurance programs for low-income children.
- MEPS data were used to formulate Recommendations in the Institute of Medicine Report—Crossing the Quality Chasm: A new Health System for the 21st Century.
- MEPS data were used to formulate recommendations in the Institute of Medicine publication on the National Quality Report.
GPRA Goal 4—FY 2002 Indicators
|Goal 4 Objectives
||FY 2002 Indicator
|Objective 4.1: Evaluate the impact of AHRQ sponsored products in advancing methods to measure and improve health care.
Fund evaluation of the impact of the CERTS program in disseminating information regarding therapeutics to at least 3 health care providers or others in order to improve practice.
As a pilot approach to program evaluation, report on a citation analysis of results of one major research initiative to assess productivity and potential impact.
Fund evaluation of private sector use of AHRQ findings to identify at least 5 private sector uses of AHRQ findings, and describe any assessment of the impact on clinical practice and/or patient care.
Fund evaluation of the number and types of patient safety events reported to system grantees.
Fund the interim evaluation of the extent to which patient safety best practices, identified in July 2001 EPC report, have been adopted by health care institutions.
Report on the evaluation of HCUPnet and the HCUP Central Distributor and implement appropriate changes to each based on evaluation results.
Fund evaluation of how AHRQ funded and conducted research on low-income populations is used by policymakers.
Initiate evaluation of the effects of AHRQ's investment in Practice-Based Research Networks.
Evidence-based Practice Centers
Use of evidence reports and technology assessments to create quality improvement tools in at least 10 organizations.
Findings from at least 3 evidence reports or technology assessments will affect State or Federal health policy decisions.
Use of evidence reports or technology assessments or access to NGC site informed organizational decision making in at least 4 cases and resulted in changes in health care processes, quality, or health outcomes.
User's panel (with composition similar to that of September 2000 patient safety research summit) will evaluate progress on patient safety research agenda defined by September 2000 summit. Evaluation criterion will be progress made in at least 50% of the research priorities as a result of the Agency's FY01 RFAs.
Establish continuous quality improvement systems for conducting process and outcome evaluations of new training and career development initiatives to help to ensure they are responsive to emerging needs and demands of the health care delivery system.
|Objective 4.2: Evaluate the impact of MEPS data and associated products on policymaking and research products.
Have a fully functional MEPS-based MEDSIM model to allow simulation of the potential impact of programmatic changes in health care financing and delivery Dec 2002.
Produce baseline FY statistics on number of MEPS-based articles published in peer review journals.
Conduct customer satisfaction survey for MEPS workshop participants to assess how MEPS data is being used to inform research and public policy.
Develop marketing plan to promote the MEPS-IC data to state officials Dec 2002.
Increase by 15% over FY 2001 baseline the number of examples of how research using MEPS has informed decisions by Federal, state and private sector policymakers.
Select for Text Version
Return to Contents
GPRA Goal 5: Support of Initiative to Improve Health Care Quality Through Leadership and Research. (HCQO)
Strategy: Quality Interagency Coordination Task Force (QuIC)
The President mandated the establishment of the Quality Interagency Coordination Task Force (QuIC) as a vehicle for promoting collaboration among the Federal Agencies with health care responsibilities to improve the quality of care in America. The Secretaries of Health and Human Services and of Labor co-lead this activity and the AHRQ Director serves as operating chair. The QuIC is working to improve patient and consumer information, quality measurement systems, the workforce's ability to deliver high quality care, and the information systems needed to support the analysis of the care provided. Input gathered through these coordinated activities contributes significantly to the development of quality-related research conducted and sponsored by AHRQ.
Types of Indicator: Process and output.
Use of Results by AHRQ:
The QuIC provides AHRQ with opportunities to further two major Agency goals. (1) In working with the Federal agencies that provide and/or purchase health care for millions of Americans, AHRQ is learning what major users of health services research on quality, evidence-based medicine and other topics need. This provides AHRQ with an invaluable source of real-time user input and directly influences the Agency's research agenda and product development. (2) The QuIC provides AHRQ with unparalleled opportunities to advance its Translating Research Into Practice agenda. The Agency is able to inform the Federal health care community about the existence of research and products that currently are in the portfolio and are relevant to the issues the community is wrestling with.
The results for these indicators are largely completed work products and success in meeting project milestones. As the Director of AHRQ is the QuIC operational chair the AHRQ Coordinator for Quality Activities is assigned to monitor progress of the various workgroups and maintains to all the pertinent data. The majority of the work products of the group are available upon completion to the public. In February 2000, the QuIC Web site became operational at www.QuIC.gov.
GPRA Goal 5—FY 2001 Results
Objective 5.1: Conduct Research to Help to Measure the Current Status Health of Care Quality in the Nation.
1st Indicator: QI Taxonomy meeting held under the auspices of the QuIC.
Results: A taxonomy is being published in a peer-reviewed journal in cooperation
with CMS on quality improvement efforts.
2nd Indicator: Number of grants and contracts funded in FY 2001 that will help to
fill gaps in the information available to assess the national quality of care, or will help to expand the use of current measures to provide a broader or richer picture of quality.
Objective 5.2: Facilitate use of quality information to improve health care in the
1st Indicator: Number of grants to improve patient safety.
Results: A complete list of grants designed to improve patient safety can be found in the appendices of this document. The following provides a summary of the types and number of grants approved for funding in FY 2001:
- Funded 17 DCERPS
- Funded 3 Centers of Excellence in Patient Safety Research
- Funded 16 event reporting demonstrations
- Funded 6 cooperative agreements to provide dissemination and education in patient safety.
- Funded a coordinating center to assist in the dissemination of research results and best practices.
2nd Indicator: Adoption of Agency sponsored research and tools developed by one or more users to facilitate consumers/purchaser/decision- maker use of information about quality.
Results: CDC is now using NIS data (from HCUP) instead of NHDS data. Two articles are to be published in the Nov. 2001 issue of IMWR based on NIS data. In addition, an AHRQ grantee, a researcher at Dartmouth, is working for CMS (formerly HCFA) on the proportion of procedures performed for Medicare patients compared with all other patients. The researcher is studying the volume-outcome relationship in the Medicare population for a number of procedures. This is the initial information AHRQ will be providing as part of an investigation into whether CMS should publicly report information on hospital volume.
Objective 5.3: Improve quality measurement.
1st Indicator: Identification of collaborators for research projects on electronic
medical records integrated with guidelines (e.g., from the Guideline Clearinghouse) or QI indicators (e.g., CONQUEST, QI Taxonomy project, HCUP measures).
Results: AHRQ has extended its collaborative partnerships throughout the country. The agency has been very successful in its efforts to collaborate on projects related to integrating guidelines from funded research with electronic medical records. The following grants that address implementation of guidelines, quality improvement indicators, or performance measures through the use of electronic medical records have been awarded by AHRQ:
- Brigham & Women's Hospital (U18 HS11046)
Improving Quality with Outpatient Decision Support
- Baylor College of Medicine (P20 HS11187)
Baylor Practice-Based Research Network
- Children's Hospital Boston, Ma. (R01 HS09390)
Family Linkages Supporting Hyperbilirubin Guidelines
- Duke University Medical Center (R01 HS09436)
Interactive, Guideline-Based Decision Support on the Web
- Indiana University (P20 HS11226)
An Inner-City Primary Care Research Network
- Medical University of South Carolina (U18 HS11132)
Primary and Secondary Prevention of CHD and Stroke
- University of Pittsburgh (R01 HS09421)
Depression Care Using Computerized Decision Support
2nd Indicator: Adoption of Living With Illness children's health measure by NCQA.
Results: In FY 2001, NCQA adopted a measure found in the CAHPS 2.0 related to living with illness for children's health. In addition, AHRQ has worked with accrediting bodies to:
- Establish a process for reviewing new surveys proposed for inclusion in the set of CAHPS surveys.
- Maintain an AHRQ representation on key external bodies including the NCQA Committee on Performance Measurement, JCAHO Board of Commissioners and Performance Measure Advisory Committee, the Consortium (formerly known as AMAP), and the National Quality Forum research subcommittee.
Return to Contents
Proceed to Next Section