Outcomes of Pharmaceutical Outcomes Research: Impact of Studies Funded
Key Impacts and Findings by Project
This section provides an overview of each of the 18 COER OPT projects, along with a description of the project's impact by type, or level, of impact.
In Level I (impact on further research) findings are presented within the following categories, as appropriate:
- Treatment effectiveness.
- Cost and economics.
- Tools for patient management.
- Research tools and translating research for clinical care practice.
- Special needs of target populations.
- Public health and prevention, and chronic and persistent diseases and conditions.
These summaries are based on review of study-related publications, as well as interviews with some principal investigators and users of the research results.
Comparative Outcomes of Ambulatory Pharmaceutical Agents
Principal Investigator: Joseph Lau, M.D.
Grant Number: HS07782
Project Period: March 1993-February 1997
This project advanced the analytical tools available to translate the results of randomized controlled trials (RCT) into clinically useful information. In addition, the project applied these tools to a broad range of key clinical conditions (e.g., renal disease, AIDS, hypertension, serious infection). Research efforts related to this project resulted in:
- New information being incorporated into clinical guidelines for chronic kidney disease patients.
- Information supporting the development of new diagnostic approaches related to acute sinusitis.
- Information supporting alternative treatment patterns for serious infections.
Methodologically, the project also advanced the science of meta-analysis, in particular, enhancing the value of relatively small, randomized, controlled trials.
The project developed a computer-based automatic process called Real-Time Meta-Analysis System (RTMAS). This process organizes, routinely updates, and displays randomized controlled trials (RCTs) and meta-analyses of pharmaceuticals in a simple and easy-to-understand matrix format. This significantly reduces the time required to conduct a systematic review and/or meta-analysis of the effectiveness or comparative effectiveness of therapeutic agents. A major contribution of the project is in the methodological improvement of meta-analysis with the development of control rate meta-regression.
Research related to this project resulted in 38 articles to date, including seven articles in the Annals of Internal Medicine, five in the Journal of Clinical Epidemiology, two in The British Medical Journal, and one in the Journal of the American Medical Association.
Impact on Clinical Practice (Level III)
Hospitalization and Infection: A meta-analysis conducted with the RTMAS confirmed that a single dose of aminoglycosides is as effective as and is less toxic than the standard multiple doses. A topic ripe for investigation, this study was one of the earliest of a number of studies in the field that confirmed this finding. According to the Project PI, anecdotal evidence suggests a definite shift in treatment patterns towards an increase in single-dose treatment as a result of this finding.
The meta-analysis compared a once-daily regimen versus the standard practice of multiple daily doses of aminoglycoside for serious infections. No significant difference was found in efficacy between the strategies in patients with serious infections, and there was a reduced risk of nephrotoxicity with the once-daily regimen. "Once-daily dosing can reduce the cost of administration and the toxicity of aminoglycoside in the inpatient setting. In addition, the once-daily dosing regimen could result in increased "ambulatory" use of these potent antibiotics instead of current practice of primarily inpatient use. The adoption of this approach has the potential for lowering costs for patients by obviating or shortening their lengths of hospitalization."3
Impact on Policies (Level II)
As further described below, work conducted under this project was influential in the development of guidelines for chronic kidney disease, recommendations for the management of acute sinusitis, a framework used by researchers in reporting RCT results, and syntheses of research on congestive heart failure and hypertension.
- National Kidney Foundation's Evidence-based Guidelines: Work conducted under this project related to non-diabetic renal disease provided important contributions to the National Kidney Foundation's Kidney Disease Outcomes Quality Initiative (K-DOQI). K-DOQI is expanding into chronic kidney disease and looking at management of lipid abnormality in chronic kidney disease patients, and the management of hypertension in chronic kidney disease patients.4
- Evidence-based Practice Center Report on the Management of Acute Sinusitis: This project's work related to antibiotics for sinusitis was integral to the recommendations made in an Evidence-based Practice Center report on the management of acute sinusitis.4
- Clinical Trials Checklist: A checklist was developed, proposed and refined by the Working Group on Recommendations for the Reporting of Clinical Trials in the Biomedical Literature. It may be used to help improve the reporting of clinical trials by: serving as a framework for authors in submitting reports on clinical trials to journals; being used as a quick reference tool in planning reports to facilitate greater uniformity and completeness; serving as a teaching tool; and offering more effective peer review.5
- Systematic Reviews by Cochrane Collaboration Collaborative Review Groups: The study potentially addresses Level II through its impact on key change agents in the field of evidence-based medicine and clinical research: "The RTMAS approach... has proven to be well-received." It was presented to key organizations, including the Society for Clinical Trials, the Cochrane Collaboration, and the American Statistical Association. The registries of congestive heart failure clinical trials and hypertension RCTs were contributed to the Cochrane Collaboration Collaborative Review Groups on these topics. Further, the project's various disease- and drug-specific meta-analyses have the potential to influence changes in clinical practice and patient outcomes in areas including treatment of hypertension, HIV/AIDS, and serious infections and in the use of aspirin.6,7
Impact on Further Research (Level I)
Both the RTMAS model itself, as well as applications of the model, have resulted in significant findings furthering research in a wide range of areas.
Research Tools and Translating Research for Clinical Care
Meta-analysis: As mentioned above, the RTMAS offers a methodological improvement of meta-analysis through the development of control rate meta-regression. This latter methodological innovation was supported in a well-recognized study on the discrepancies between meta-analysis and a large RCT. Another methodological area explored was the comparison of meta-analysis of large studies versus meta-analysis of small studies; the results of smaller studies are usually compatible with the results of large studies, but discrepancies do occur even when diversity across studies is considered. Examining sources of heterogeneity between the results of large and smaller trials provides information that may not be conveyed by estimates of average treatment effect. Further, the investigators identified the need to "to re-examine the idea that we should not start a randomized control trial unless sufficient patients are available to avoid reasonable Type I and II errors. Meta-analyses will come to the rescue, provided trials continue to be published at the present rate."4
The project addresses Impact Level I in that RTMAS provides a unique and powerful way of visualizing the availability of evidence. The ability to dynamically generate evidence tables by answering a series of questions makes it more efficient to conduct systematic reviews. From a collection of studies of various designs, RTMAS could readily determine the quantity of various levels of evidence available for synthesis and decide whether certain questions can be answered.4
Bringing Practice Up to Speed with Research: Comparison of the results of RCTs with clinical expert recommendations reveal that these recommendations are not often synchronized with accumulating evidence. The rational use of therapeutic drugs can be achieved only through the routine use of meta-analysis on high-quality clinical data. This implies that clinical trials related to the same topic should be viewed as a continuum of experience; meta-analyses should be revised regularly; and a method of regularly updating and reporting findings to individual physicians and investigators should be developed.
Proposed steps to pursue include ensure all clinical trials follow an agreed-upon scientific format; set up a mechanism for registering all clinical trials undertaken; ensure that all registered clinical trials with some outcome data are included in ongoing cumulative meta-analyses; ensure that experts trained both in the specialties of interest and in clinical trials monitor databases of earlier noted recommendations, and agree on information to go to health practitioners; and institute regular physical and knowledge examinations for physicians.6
- ACE Inhibitors and the Progression of Non-diabetic Renal Disease: This study found that Angiotensin-converting enzyme (ACE) inhibitors are more effective than other anti-hypertensive agents in reducing the development of end-stage non-diabetic renal disease, and they do not increase mortality.8
- Diabetes and Non-diabetic Renal Disease: Pooling the results of 10 studies in 2 meta-analyses showed that dietary protein restriction effectively slows the progression of both diabetic and non-diabetic renal diseases. These results provide sufficient justification to recommend dietary protein restriction for well-informed patients with chronic renal disease and renal insufficiency.9
- Antibiotics for Acute Sinusitis: This study found that amoxycillin and folate inhibitors are essentially as effective as more expensive antibiotics for the initial treatment of uncomplicated acute sinusitis. Small differences in efficacy may exist but are unlikely to be clinically important.10
- HIV Infection: Sulfamethoxazole-trimethoprim is the superior regimen, and low doses could improve tolerance without losing effectiveness for primary prophylaxis. Low doses of dapsone reduce toxic effects, but at the expense of some loss of efficacy. RCTs may underestimate the case-fatality of Pneumocystis carinii pneumonia (PCP) in the community where surveillance is less rigorous and diagnosis is often delayed. In addition to preventing PCP, oral regimens were shown to reduce the incidence of toxoplasmosis by an on-treatment analysis.11
- HIV Infection: For patients with AIDS and expected survival greater than 3.8 years, low dose trimethoprim-sulfamethoxazole (TS) is better than standard dose TS as the first choice agent for preventing PCP. As patients with AIDS live longer, the routine use of low dose TS will be more than adequate for patients at risk for PCP.12
- HIV Infection: Mortality rates are halved by combinations involving protease inhibitors or lamivudine, at least in the short term. Switching to or adding didanosine, zalcitabine or stavudine in patients on long-term azidovudine results only in modest risk reductions in disease progression. Changes in therapy can delay disease progression and improve survival in antiretroviral-experienced patients and may be relatively more effective in earlier stages.13
- Hypertension in the Elderly: One study assessed the effect of anti-hypertensive drug treatment on mortality and morbidity in elderly patients and found that treatment of hypertension in this population produces a significant benefit in total mortality and cardiovascular morbidity and mortality.14
Public Health and Prevention, and Chronic and Persistent Disease
Gastrointestinal and Hepatic Diseases: This analysis finds that:
"RCTs are so necessary to evaluate the small incremental changes in the treatment of gastrointestinal and hepatic diseases evolving out of basic research that their future potential is enormous. As the suggested improvements in treatment become smaller, and the use of a no-active-treatment group less ethically acceptable, individual trials will seldom be large enough. Cumulative meta-analyses will be increasingly important as one contributor to the decision needing to be made in a timely fashion as to whether a new treatment is better, worse or the same as the standards. Acute viral hepatitis is so variable in its course that RCTs will be crucial to the proper evaluation of the many new antiviral agents coming out of the laboratory in the next few years."7
The study provides suggestions for future research directions, including selective RCTs that focus on specific treatment issues e.g., "the presently popular proximal or selected vagotomy for peptic ulcer is not entirely satisfactory and many attempts at improvement can be expected.... Also, the five methods of treating bleeding peptic ulcers through the endoscope need to be compared in well-controlled studies."7
Computer-based Prospective Drug Utilization Review
Principal Investigator: William M. Tierney, M.D.
Grant Number: HS07763
Project Period: July 1993-January 1997
In this project, the investigators incorporated treatment guidelines into a computer-based system in a large institutional setting where all test and prescription ordering must be carried out on computer stations. The randomized controlled trial was conducted to demonstrate the impact of this system on treatment patterns, prescriptions, and patient-centered outcomes, but concluded without positive result. However, the research resulted in significant lessons about:
- Pharmacist work patterns.
- The importance of patient-reported symptom information.
- Providing computer-based information to physicians so that it is effective in influencing decisionmaking.
- Using computerized patient databases to identify patients most in need of intervention.
The project helped to fuel research in other areas such as aging and palliative care. In addition, the project resulted in the adoption of new policies and procedures for the large integrated delivery system in which the study was conducted. This health services delivery system continues to attract attention for its innovations in system-wide and centralized computerization for clinical care.
Impact on Further Research (Level I)
Cost and Economics of Health Care
- Using Electronic Medical Records to Predict Mortality: The purpose of this study was "to identify high-risk patients with heart disease by using data stored in an electronic medical record system to predict six-year mortality." The study found that routine clinical data generated during ordinary clinical encounters and stored in patients' electronic medical records "are capable of predicting mortality among patients with heart disease. This could allow increasingly scarce health care resources to be focused on those at highest mortality risk.... Unlike most previous studies which often included only referred patients, inpatients, or patients with specific types of [disease, treatment or tests], we studied all primary care patients with any evidence (correct or not) of ischemic or congestive heart disease."15
Tools for Patient Management
Computerizing Practice Guidelines: The final report for this project, as well as other project-related publications, highlight some critical lessons about when and how guidelines can most effectively be incorporated into electronic systems from the provider's perspective. The final report describes the necessary conditions for success in computerizing practice guidelines: "The guidelines must be accepted as the local standard of practice by clinicians...computer prompts must be timely and use data that are already available electronically...and computerized guidelines should have measurable effects." The study also concludes that "changing from a paper-based outpatient prescription system to a microcomputer-based prescription-writing system had profound effects on the types of activities carried out by pharmacists in a hospital-based outpatient pharmacy." The report notes "that delivering real-time drug utilization review (DUR)-based treatment suggestions to outpatient pharmacists had dramatic and salutatory effects on their [pharmacists'] work patterns, increasing opportunities for them to advise patients, consult with providers, and solve problems." With respect to physicians, the report also concludes "that the favorableness of respondents' opinions of the inpatient workstations declined as the level of training increased, a trend that was independent of computer literacy. Increasing computer use by physicians would probably require modification of the educational and socialization process rather than mere reliance on increasing computer literacy."16
A related article finds that the AHCPR Heart Failure Guidelines as written cannot be automated in existing electronic medical record systems. The author proposes the following recommendations to improve future versions of these and other clinical practice guidelines:
Write all guidelines in a simple "if-then-else" format; make algorithm logic hinge on explicitly defined values of accepted clinical parameters; expect that local translation of guidelines will be necessary and help guide that process; include rules about errors of commission and omission, and don't ignore common co-morbid conditions; balance the costs of diagnosis and treatment (not only in terms of dollars) and consider the likelihood that individual patients will benefit from the guidelines' recommendations; and evaluate the resulting guidelines using real patients and representatives of all those who will use them.17
Electronic Guideline-based Treatment Suggestions for Pharmacists: The component of the project that looked at pharmacist work patterns found favorable results: "The results of this study indicate that a dramatic, favorable change in work patterns occurred when pharmacists were provided with an electronic display of guideline-based, patient-specific treatment suggestions. Specifically, pharmacists who had access to treatment suggestions spent less of their time preparing and filling prescriptions and more of their time functioning in an advisory role with patients, physicians, and nurses. Moreover, more of the pharmacists' time was spent solving problems.... The delivery of patient-specific information to pharmacists at the time of dispensing significantly and favorably influenced pharmacist work patterns by increasing opportunities for pharmacists to advise, consult and solve problems."18
Researchers also learned about the barriers pharmacists face in talking with patients, which is important for prospective utilization review. Patients often do not like to interact with pharmacists in the drug store. Further, pharmacists may feel intimidated dealing with physicians, and prefer the less threatening E-mail response system.19
- The Importance of Symptom Information, and Quality-of-life Measures: The DUR project revealed that it may be helpful to provide physicians with additional information for decisionmaking as long as the information is provided in such a way that physicians can apply the information to each patient individually, rather than being forced to categorize patients according to a predetermined definition of disease. The study highlighted the importance of providing physicians with symptom information. Because of this, the researchers have sought to find ways of measuring symptomatic information. They explored ways of assessing clinically important changes associated with quality of life. For example, in one analysis they determined that the standard error of measurement (SEM) closely approximates minimal clinical important difference standards (MCID) for key dimensions of common health-related quality-of-life instruments.20,21
- Association of Electronic Prescription Records with Compliance: How well does electronic data on prescriptions indicate actual client use? This study examines whether the medication possession ratio (MPR) values computed from electronic prescription records at an internal medicine practice were associated with medication compliance as reported by patients for the past month using a validated questionnaire. "We conclude that our patients receive sufficient quantities of their medications, but MPR estimates vary widely. Older patients have higher medication availability. Patient responses to selected questions from validated compliance questions are associated with the MPR but differ by disease."22
Translating Research to Influence Practice
- Multidimensional Work Sampling Among Pharmacists: This study found a multidimensional work sampling methodology to be a valuable tool in gathering quantitative information about the use of human resources in the delivery of care. The study measured the baseline effect of implementing expert system technology on pharmacist work activities in a large outpatient hospital pharmacy. "Though our baseline data indicate that our pharmacists spend the largest portion of their time working alone...technologic and process changes within the pharmacy should increase opportunities for educating patients about their drugs and consulting with physicians about drug therapy." However, the "quick code" concept must be evaluated further. Codes provide pharmacists with a simple and efficient way to record frequently occurring tasks but the disadvantage is "the risk that pharmacists may gravitate towards these single-entry codes when another 3-entry combination might provide a more accurate description of what they were doing when signaled."23
Special Needs of Target Populations
- The Relationship of Race and Age to Primary Care: This study found that "African American patients had fewer outpatient encounters, including primary care visits, emergency department visits, urgent care visits, or pharmacy visits for medication refills. Among males, African Americans had greater emergency department use than white patients." The study also found that younger African Americans have a rate of hospitalization that is three to six times greater than 30- to 35-year olds, with the greatest risk for African American males. The authors conclude that inadequate primary care among inner city adolescents and African Americans with asthma results in costly and morbid hospitalizations.16
Improving Outcomes in Elderly NSAID Users
Principal Investigator: Wayne A. Ray, Ph.D., M.S.
Grant Number: HS07768
Project Period: July 1993-July 1998
This project addressed the use of non-steroidal anti-inflammatories (NSAIDs) for osteoarthritis in the elderly population. The project:
- Resulted in published guidelines for the management of osteoarthritis, including preferred drug treatment.
- Demonstrated the value of prior authorization in reducing costs for NSAIDs, which have similar efficacy but substantial variation in cost.
- Provided guidance in the use of administrative data for research purposes.
Impact on Policies (Level II)
- Guidelines and Management for Osteoarthritis: The objective of several publications in leading clinical journals was to guide primary care physicians in their approach to patients with osteoarthritis and other disorders causing musculoskeletal pain. In most patients, acetaminophen is preferable to NSAIDs as a first-choice agent because of its lower toxicity. If NSAIDs are used they should be prescribed initially in lower doses. Ibuprofen may be the preferred initial NSAID.24,25
Impact on Further Research (Level I)
Studies associated with this project identified the significant cost savings that can result from prior authorization features of drug utilization review. It also illustrated the advantages and challenges of using administrative data (e.g., Medicaid data) for research purposes.
Cost and Economics
- The Cost of NSAID Use: This project conducted a cost-identification, retrospective cohort study among Tennessee Medicaid enrollees age 65 years or older that quantified utilization of and costs for medical care for treatment of gastrointestinal disease attributable to non-steroidal anti-inflammatory drug (NSAID) use. NSAID use in elderly patients was associated with substantial excess costs and utilization of medical care for gastrointestinal disorders. The study found that prior-authorization requirements may be highly cost effective with regard to expenditures for NSAIDs, drugs that have very similar efficacy and safety but substantial variation in cost. Expenditures decreased by 53 percent for estimated savings of $12.8 million. The reduction in expenditures resulted from the increased use of generic NSAIDs as well as from a 19 percent decrease in overall NSAID use. There was no concomitant increase in Medicaid expenditures for other medical care. Regular users of non-generic NSAIDs had similar reductions in NSAID expenditures and use, with no increase in expenditures for other medical care.26,27
Translating Research to Influence Practice
- Use of Administrative Data: "If used properly, administrative databases are a valuable resource for retrospective evaluations.... However, the usefulness of these databases can be limited by data quality, the absence of a control group, the lack of data elements that correspond to important outcomes, and problems in accounting for differences in case mix. Including more data elements in administrative databases is an obvious approach to improving their suitability for use in research. Ultimately, greater emphasis needs to be placed on concurrent, planned evaluation of policy and program changes through either randomized controlled trials or prospective cohort analyses."28
Outcomes Associated with Therapy for Otitis Media
Principal Investigator: Stephen Berman, M.D.
Grant Number: HS07816
Project Period: July 1993-June 1997
This study focused on the comparative cost-effectiveness of various antibiotic treatments of otitis media (OM). The investigators used the Colorado State Medicaid database to identify exposure and outcomes. Information related to this work has been used by a variety of organizations in development of policies and educational programs. In addition, the study identified important issues for further research in the area of diagnosis and treatment of the condition.
Impact on Policies (Level II)
Dr. Berman has worked with the Colorado State Department of Health to develop guidelines for Colorado Medicaid. He has also worked with the American Medical Association on a program to evaluate outcomes of otitis media that will be pilot tested in Utah. He has developed an Internet-based interactive Web site to teach current guidelines on the treatment of OM. The site and a companion workshop are sponsored by the American Academy of Pediatrics. Dr. Berman reports working with a number of managed care companies to develop guidelines but impact has not been quantified. Finally, Health Plan Employer Data and Information Set (HEDIS) measures include the use of amoxicillin or trimethoprim/sulfamethoxazole (TMP/SMX) as initial therapy for otitis media.29
Concern about the development of resistant strains of bacteria have led a number of key public health policy and delivery organizations—such as the Centers for Disease Control and Prevention, State health departments, and HMOs—to become interested in this research, because OM is one of the primary conditions for which antibiotics are prescribed. Researchers involved with the project noted that it is important that public agencies continue to support medical outcomes studies that provide a balanced perspective on such things as the use of limited spectrum versus broad spectrum antibiotics when appropriate, including for otitis media.
Impact on Further Research (Level I)
The findings of this project not only suggest that physicians over-prescribed antibiotics, and often failed to obtain appropriate audiology testing, but also show that the more expensive antibiotics were not associated with better outcomes.
- Otitis Media: This study provides information on variations in prescribing patterns by specialty of physician and by treatment setting, as well as cost and outcomes data associated with different drug treatments. Physicians overprescribed antibiotics, often failed to obtain appropriate audiology testing, and referred one in five of the children who had ventilating tubes prior to the recommended time of 3 months. While there was a wide variation in the selection of antibiotics to treat otitis, the more expensive antibiotics were not associated with better outcomes. Further, differences exist between family physicians and pediatricians in practice patterns related to managing persistent and recurrent otitis media. Family physicians would refer children for ventilating tube surgery three times more often than pediatricians when an asymptomatic middle ear effusion is present. The study also notes that to provide population-based information there is a need to measure the volatility of enrollment in an insured population before calculating rates of utilization.30,31,32
Cost and Economics
- The Cost of Antibiotics for Treating Otitis Media: Thirty percent of children treated for new-onset OM in Colorado's fee-for-service Medicaid program received expensive antibiotic therapy, rather than less costly, equally efficacious products. More expensive antibiotics accounted for approximately 77 percent of the dollars spent for medications to treat otitis media in this population, but were not associated with better outcomes. Per child with otitis, expenditures were higher for males than for females, and higher for Caucasian children compared to Hispanic children or African American children.33
The Implications of the Cost of Otitis Media Treatment for Future Vaccination: This analysis developed information about OM-related medical expenditures to help evaluate the cost-effectiveness of vaccines being developed for respiratory pathogens that cause OM. It used data on medical visits, pharmaceuticals and surgical procedures for 87,057 children 13 years of age or younger who were continuously enrolled in Colorado's fee-for-service Medicaid program during 1992.
Findings showed that expenditures for visits are the largest expenditure component, therefore case management guidelines should emphasize reducing unnecessary visits by improving physician training in pneumatic otoscopy and by scheduling followup visits for asymptomatic children 3 to 4 weeks after diagnosis rather than 10 to 14 days. Further, because 40 percent of expenditures are incurred between 1 and 3 years of age, vaccines designed to reduce the incidence of otitis media are most likely to be cost-effective if they can be administered before the child's first birthday.34
Tools for Patient Management
The Effect of Continuous Medicaid Enrollment and Having an Assigned PCP: This analysis looked at the likelihood of a Medicaid child enrollee having a primary care physician (PCP) if he or she were continuously enrolled in Medicaid as compared to discontinuously enrolled. It also looked at the relationship between having a PCP and emergency room use for otitis media.
Children and continuously enrolled in Medicaid throughout the entire year were more than four times as likely to always or sometimes have a PCP compared with children who were discontinuously enrolled. Further, the likelihood of ever using the emergency department for an otitis media-related visit was increased by 26 percent when a child sometimes had a PCP, and increased by 50 percent when a child never had a PCP. Finally, the likelihood of ever filling an antibiotic for otitis media was reduced by 23 percent and 34 percent respectively when a child sometimes or never had a PCP compared with always having a PCP.35