Translating Evidence into Practice 1998 (continued, 7)
Session 11. Using Information as a Tool for Change
Moderator: Elinor Walker, AHCPR
Integration of Guidelines into Real Time Clinical Tools—David Shriger, M.D., M.P.H., Associate Professor, Emergency Medical Center, UCLA Medical Center
Dr. Shriger's project was developed under a grant from AHCPR to determine how to modify physician behavior in the emergency room. The challenge of the project was to deliver high-quality emergency care in the hospital despite turnover, a large number of complaints, time pressure, and the lack of an established patient/provider relationship.
The project's goal was to provide assistance at the time the physician is seeing the patient in real time, in the normal flow of care. The plan also included giving the provider a reward for participating. The provider must have an unambiguous indication of what to do and the flexibility to deviate from the information or diagnosis provided.
The Emergency Department Expert Charting System (EDECS) was tested on five common complaints seen in the emergency room: occupational exposure to body fluids (e.g., needlestick punctures), fever in children under than age 3, low back pain, male urogenital problems, and epileptic seizures. EDECS improved the documentation and the appropriateness of testing, as well as the appropriateness of treatment with the EDECS program. On the negative side, internal and external validity could be potential study flaws in this project.
The ABCs of Benchmarking: Achievable Benchmarks of Care—Norman W. Weissman, Ph.D., University of Alabama
The Achievable Benchmarks of Care (ABC) project is a 5-year project funded by one of eight Q-SPAN initiative grants for advancing the science of measurement of the quality of care. ABC is an interdisciplinary effort and a collaboration between an academic medical center and others in the health care sector. Dr. Weisman said the project is grounded in the belief that providing specific benchmarking and feedback will change processes of care.
Using the ABC levels to set benchmarks, top providers and their case outcomes are identified and pooled to be analyzed. All providers with high performances contribute to the analysis. Providers with small numbers of cases are tagged and affect the benchmark level the least. The ABC is then calculated using the paired-mean method.
The ABC method provides a private, nonthreatening analysis based on multiple patients; the criteria are clearly defined and the directions for improvement are clear in the feedback process. However, Dr. Weissman said the current ABC process can be improved with more timely feedback and larger patient samples. The method should be applied to the most immediately actionable aspects of care to bring about change.
Final Plenary Session
Evidence-based Medicine: Academic Jobs Program or a Solution to Cost Quality Challenges Facing Medicine—Eric B. Larson, M.D., Medical Director, University of Washington Medical Center
Dr. Larson stated that evidence-based medicine and practice guidelines are assumed to be solutions to improving health care quality and reducing costs. The evidence-based medicine guidelines and conclusions, however, are susceptible to manipulation, and critical syntheses, reviews, and meta-analyses are susceptible to bias. Organizational and managerial factors may have a more powerful effect on outcomes than evidence-based medicine guidelines.
Evidence-based medicine is an important information service for professionals and the public. Evidence-based medicine can also be used to expose the professional and the public to difficult cost/quality trade-offs. The advantages of Web technology will help spread evidence-based medicine to the public. The public is often misled when evidence-based medicine is advanced as a solution to the problems of rising costs, by rising public expectations, and by limitless technologic advances. It is the combination of the evidence-based medicine movement and the incorporation of public values that offers the best solution to the cost-quality tradeoff.
Problems in the "Evidence" of Evidence-based Medicine—Alvan Feinstein, M.S., M.D., Sterling Professor of Medicine and Epidemiology, Yale University School of Medicine
Dr. Feinstein began with a case report of a 78-year-old widower who lives with his children in a reasonably good functional state but presents with illness. Many common questions have to be asked about treating this man; how does evidence-based medicine answer these decisions? The basic tenets of evidence-based medicine consist of integrating individual clinical experience with the best available external clinical evidence from systematic research. What is new is a textbook of clinical practice, the contents of which have been offered as evidence-based medicine. The text says we should try to avoid the nonexperimental approaches because they routinely lead to false positive conclusions about efficacy. The randomized trial, and especially the systematic review of several randomized trials or meta-analyses, has become the gold standard.
Several publications have stated that medical practice is evidence-based. Looking at how well randomized trial evidence works in practice, Dr. Feinstein noted that there are considerable restrictions on data from randomized trials and most of the soft clinical data are omitted. The patient's expectations and desires are almost invariably omitted from the data of randomized trials. In randomized clinical trials, using a so-called unbiased statistical analysis—the intention to treat—no attention is given to anything that happens after randomization.
Medical students and doctors are not offered instruction on pathophysiologic and other judgmental reasoning. There is no incentive to learning how to think. Important new discoveries not covered in randomized trials may be disdained. Results may be used improperly by the purchasers and managers to cut the costs of health care. Authorization or a payment may be denied for anything not supported by evidence-based medicine. Individual meta-analyses may not always be complete or updated.
Update: Evidence-based Practice Centers™ (EPCs), Task Force on Preventive Health Services, and Putting Prevention into Practice—Douglas B. Kamerow, M.D., M.P.H., AHCPR, and David Atkins, M.D., M.P.H., AHCPR
Dr. Kamerow said the Evidence-based Practice Centers, created by AHCPR in 1997, are to work with the public and private sector to help create evidence-based reports and technology assessments and then launch them for implementation. All of this activity is directed toward improving the quality of clinical care in this country through evidence-based practice.
The reports will be used in clinical practice guidelines, performance measures, other kinds of quality measures, and quality improvement programs, as well as to help influence coverage policies. The centers will also produce cost-effectiveness analyses and decision analyses, and they will undertake methodologic research. The reports will be widely published including on the Web. The reports will include evidence tables and summaries, extensive bibliographies, and often a research agenda.
Dr. Atkins stated that the Preventive Services Task Force has gone through two iterations of the Guide to Clinical Preventive Services and released the last edition of the Guide in 1996. Now, the Guide will be completely revised. Controversial aspects have centered on recommendations to not do certain services that are widely used. The next Task Force will have the support of two EPCs at Research Triangle Institute/University of North Carolina-Chapel Hill and the Oregon Health Sciences University. The aim will be to produce updated reports that will be put online or published as journal articles and will be coordinated with the ongoing work of Put Prevention Into Practice. The first job of the Task Force will be to identify new topics or priority updates with input from experts and stakeholders.
Put Prevention Into Practice (PPIP) is an implementation program to address the barriers to preventive care and included the Clinician's Handbook of Preventive Services, more user-friendly guides, and office materials. Materials are now available over the Web. CDC's Preventive Services Priorities Project looked at Task Force recommendations and ranked them based on preventive burden of disease and cost-effectiveness. This project found that services can be separated as high- and low-priority issues, and physicians will have to focus on high-priority services, realizing they cannot have it all.
Lisa A. Simpson, M.B., B.Ch., Deputy Administrator, AHCPR
Dr. Simpson stated that the focus of the conference was translating evidence into practice. The word "translation" focuses us on a bridge that goes over the gaps between knowledge and practice, rhetoric and reality, methods research and methods practice, researchers and practitioners, change and sustainability, and success and replication.
To face the challenges of change, Dr. Simpson suggested that we have to contend with clinical judgment and knowledge. We find it difficult to change: all improvement in care is change, but not all change is improvement.
For translating evidence into practice, some things work for some of the people, some of the time. The better questions are: what works, under which circumstances, in what setting, and for which populations. These drive AHCPR's agenda for the future.
In terms of goals, AHCPR's have been outcomes, quality, cost-use and access. More recently, we have focused on knowledge development (emphasizing quality and cost). We have been taking knowledge to create tools for decisionmakers, whether they are guidelines, evidence reports, quality measures, consumer surveys, and instruments for measuring outcomes. In the future, Dr. Simpson emphasized, we will need more dissemination, evaluation, and implementation of this knowledge and these tools.