Commentary-From the Clinics to the Courts: The Role Evidence Should Pl
The movement toward evidence-based medicine is a bit overdue. As noted above, until quite recently scientific research has focused mainly on costly new technologies, to the relative neglect of ordinary care and its outcomes. And even where good studies are available to guide care, physicians' clinical practices have sometimes been woefully out of step.20 Underuse, overuse, and misuse of diagnostic and therapeutic modalities are now recognized as a major problem throughout medicine (Chassin and Galvin 1998), as are avoidable errors that lead to morbidity and mortality (Institute of Medicine 1999). Even care whose well-documented value should be obvious, like aspirin after a heart attack, is often neglected (Newcomer 1998; Donohoe 1998). Thus the bare fact that physicians commonly practice or accept a particular pattern of care unfortunately tells us too little about whether that pattern is salutary for patients, let alone affordable for plans. As noted by Havighurst et al., "courts' continued focus on professional custom for setting the legally recognized standard of care reinforces this disregard of scientific progress" (see the Introduction to this special issue by Clark C. Havighurst, et al.).
The poor correlation between what physicians actually do, and the emerging evidence showing what they should do, is just one of several reasons why courts should no longer determine what physicians ought to do by looking primarily at what they actually do. In addition, many prevailing practices often do not even reflect what physicians believe they should do. Defensive medicine, patients' demands for the latest technologies, physicians' own fascination with new devices and procedures, and economic pressures can quickly skew physicians' most thoughtful, scientifically well-founded preferences. Beyond this, physicians are neither morally nor financially entitled to do literally whatever they ideally wish to do for their patients. Health care is costly, and the days of handing physicians a blank check to spend unlimited amounts of other people's money are over (Morreim 1997: 16-27).
Accordingly, the courts should move away from their tradition of setting physicians' professional standard of care by referring to their prevailing practices—even with the usual caveats for differing schools of thought, reputable minorities, and the like. Instead of asking for evidence about what members of the profession (say they) do, courts should begin, just as they should with health plans, to expect evidence of what actually works for patients. Indeed, this expectation of reliable scientific evidence rather than "general practice or acceptance" is precisely the transition that the Supreme Court has asked courts to make, in the move from Frye to Daubert, as Shuman indicates in this issue. Although this move was not entirely feasible for health care only a few years ago, the emergence and improving methodologic quality of outcomes research now makes such a transition increasingly plausible and desirable. Hence if, per Shuman's analysis, courts have been reluctant to bring Daubert standards to medical litigation, it is time to begin that admittedly difficult transition.
This is not to say that courts should look exclusively at scientific evidence in judging physicians' performance. For one thing, it was noted above that outcomes research and technology assessment, and the guidelines to which they sometimes lead, still need considerable improvement and expansion, and they must be kept up to date as technologies and information change (a daunting if not virtually impossible task). Hence, flexibility is at least as important here as in the assessment of health plans' performance. For another thing, even the best CPGs cannot possibly dictate each patient's course of care. They are based on generalities that hold true on average, but have only limited room to accommodate the natural variations among individuals in any population.21 Moreover, they do not accommodate important patient preferences that can thwart even the best treatment plans. Thus good medicine unavoidably requires sophisticated professional evaluations of patients' signs and symptoms, and careful judgments to determine whether this patient's situation is sufficiently typical to warrant following rather than deviating from the applicable CPG.
The upshot of these observations is that, while courts should expect various CPGs to be defended as scientifically reasonable, even though not demanding lockstep adherence to any particular CPG, nevertheless they should expect that, if a medically reasonable CPG suggests conduct from which the physician has deviated, the physician should be able to explain that deviation with something more than a flat assertion that "in my professional judgment, the guideline did not apply."
Judges will face a major challenge at this point. As noted above, CPGs abound, many of them with dubious scientific credentials, and physicians may often be right to deviate from them. At the same time, emerging outcomes research suggests that many of physicians' current clinical routines (and therefore, quite likely, many of their deviations from the CPGs that aim to improve those routines) are not scientifically adequate either. Judges will thus have a difficult time sorting out whose evidence is more credible. Fortunately, Daubert and progeny do not ask judges to determine which side is right. Juries ordinarily take a case from evidence to answers. But the task of screening the evidence will be daunting enough, as the authors in this collection have shown so well.
Medicine is in the midst of an extraordinary transition. Economic pressures, the quest for quality, and a host of other factors have finally begun to force medicine to become truer to the scientific roots it has always claimed to have. But that transition is also an upheaval. Cherished assumptions and long-entrenched habits have been castigated or supplanted, though not always with improved alternatives. And the upheaval will undoubtedly continue for years to come. If it is a good thing to bring better science to health care, then surely it will also be a good thing, at least in principle, to bring Daubert's demand for greater rigor to the legal proceedings that sort out the consequences of health care and, in many ways, shape the future of health care itself. The move toward greater rigor both in the courts and in the clinic will be difficult but ultimately should bring major improvements for the care of patients and for the fair treatment of litigants.
Avanian, J. Z., M. B. Landrum, S. T. Normand, E. Guadagnoli, and B. McNeil. 1998. Rating the Appropriateness of Coronary Angiography—Do Practicing Physicians Agree with an Expert Panel and with Each Other? New England Journal of Medicine 338:1896-1904.
ECRI. 1995. High-Dose Chemotherapy with Autologous Bone Marrow Transplantation and/or Blood Cell Transplantation for the Treatment of Metastatic Breast Cancer. Healthy Technology Assessment Information Service, Executive Briefings, February.
Ferguson, J. H., M. Dubinsky, and P. J. Kirsch. 1993. Court-Ordered Reimbursement for Unproven Medical Technology: Circumventing Technology Assessment. Journal of the American Medical Association 269:2116-2121.
Friedman, M. A., J. Woodcock, M. M. Lumpkin, J. E. Shuren, A. E. Hass, and L. J. Thompson. 1999. The Safety of Newly Approved Medicines: Do Recent Market Removals Mean There Is a Problem? Journal of the American Medical Association 281:1728-1734.
Hartert, T. V., H. H. Windom, S. Peebles Jr., L. R. Freidhoff, and A. Togias. 1996. Inadequate Outpatient Medical Therapy for Patient with Asthma Admitted to Two Urban Hospitals. American Journal of Medicine 100:386-394.
Legorreta, A. P., J. Christian-Herman, R. D. O'Connor, M. M. Hasan, R. Evans, and K. M. Leung. 1998. Compliance with National Asthma Management Guidelines and Specialty Care. Archives of Internal Medicine 158:457-464.
Morreim, E. H. 2000c. Saving Lives, Spending Money: Shepherding the Role of Technology. In Ethical Issues in Health Care on the Frontiers of the Twenty-First Century, ed. S. Wear, J. J. Bono, G. Logue, and A. McEvoy. Dordrecht: Kluwer Academic, 65-112.
Peters, W. P., and M. C. Rogers. 1994. Variation in Approval by Insurance Companies of Coverage for Autologous Bone Marrow Transplantation for Breast Cancer. New England Journal of Medicine 330:473-477.
Rowlings, P. A., S. P. Williams, K. H. Antman, et al. 1999. Factors Correlated with Progression-Free Survival after High-Dose Chemotherapy and Hematopoietic Stem Cell Transplantation for Metastatic Breast Cancer. Journal of the American Medical Association 282:1335-1343.
Stadtmauer, E. A., A. O'Neill, L. J. Goldstein, P. A. Crilley, K. F. Mangan, J. N. Ingle, I. Brodsky, S. Martino, H. M. Lazarus, J. K. Erban, C. Sickles, J. H. Blick, and Philadelphis Bone Marrow Transplant Group. 2000. Conventional-Dose Chemotherapy Compared with High-Dose Chemotherapy Plus Autologous Hematopoietic Stem-Cell Transplantation for Metastatic Breast Cancer. New England Journal of Medicine 342:1069-1076.
Weiner, J. P., S. T. Parente, D. W. Garnick, et al. 1995. Variation in Office-Based Quality: A Claims-Based Profile of Care Provided to Medicare Patients with Diabetes. Journal of the American Medical Association 273:1503-1508.
Weiss, R. B., R. M. Rifkin, F. M. Stewart, R. L. Thenaule, L. A. Williams, A. A. Herman, and R. A. Beveridge. 2000. High-Dose Chemotherapy for High-Risk Primary Breast Cancer: An On-Site Review of the Bezwoda Study. Lancet 355:999-1003.
The author is grateful to Peter D. Jacobson and to Clark C. Havighurst for their very helpful comments on earlier drafts of this manuscript.
1. Burnum 1987: 1220-1222; Hardison 1979: 193; Reuben 1984: 592; Holoweiko 1995: 180-181; Woosley 1994: 250; Avorn, Chen, and Hartley 1982: 4-8; Lange and Hillis 1998: 1839; Avanian et al. 1998: 1901.
3. For example, the Pacific Business Group on Health, a coalition of over thirty large employers, requires its health plans to meet certain quality standards, including customer service, preventive care, and data collection (see Bodenheimer and Sullivan 1998: 1003-1007; Schauffler, Brown, and Milstein 1999: 134-142).
4. For instance, treatment-related deaths (death within one month after transplant) were "frequently disregarded; these patients were reported as 'unevaluable' because they 'did not survive long enough to exhibit a clinical response.' In many cases, omitting these patients led to higher response rates. Eliminating early deaths is inappropriate (particularly when they may have been caused by the treatment) and not standard for trial design or analysis." (ECRI 1995: 7).
Other methodological problems were rampant. Some studies lacked controls entirely, others included only those patients who had already shown they were responsive to chemotherapy, and still other studies neglected to keep track of key patient characteristics, such as the number of metastatic sites or estrogen receptor status (ibid.).
5. Harris v. Mutual of Omaha Companies, 992 F.2d 706 (7th Cir. 1993); Sweeney v. Gerber Products Co. Medical Benefits Plan, 728 F. Supp. 594 (D. Neb. 1989); Thomas v. Gulf Health Plan, Inc., 688 F. Supp. 590 (S.D. Ala. 1988); Arrington v. Group Hospitalization and Medical Service, 806 F. Supp. 287 (D.D.C. 1992); Fuja v. Benefit Trust Life Insurance Co., 18 F.3d 1405 (7th Cir. 1994); Bechtold v. Physicians Health Plan, 19 F.3d 322 (7th Cir. 1994); Holder v. Prudential Insurance Co. of America, 951 F.2d 89 (5th Cir. 1992); Graham v. Medical Mutual of Ohio, 130 F.3d 293 (7th Cir. 1997); Turner v. Fallon Community Health Plan, Inc., 127 F.3d 196 (1st Cir. 1997).
6. Pirozzi v. Blue Cross-Blue Shield of Virginia, 741 F. Supp. 586 (E.D. Va. 1990); Wilson v. Group Hospitalization, 791 F. Supp. 309 (D.D.C. 1992); Calhoun v. Complete Health Care, Inc., 860 F. Supp. 1494 (S.C. Ala. 1994); Henderson v. Bodine Aluminum, Inc., 70 F.3d 958 (8th Cir. 1995); Dahl-Eimers v. Mutual of Omaha Life Insurance Co., 986 F.2d 1379 (11th Cir. 1993); Wilson v. CHAMPUS, 65 F.3d 361 (4th Cir. 1995); Bucci v. Blue Cross-Blue Shield of Connecticut, 764 F. Supp. 728 (D. Conn. 1991); Mattive v. Healthsource of Savannah, Inc., 893 F. Supp. 1559 (S.D. Ga. 1995); Killian v. Healthsource Provident Administrators, 152 F.3d 514 (6th Cir. 1998); Bailey v. Blue Cross/Blue Shield of Virginia, 67 F.3d 53 (4th Cir. 1995).
7. See, e.g., Fox v. HealthNet, No. 219692, 1993 WL 794305 (Riverside County Super. Ct./Central Cal. Dec. 23, 1993), discussed in Furrow 1997: 447; Meyer and Murr 1994; Newsweek 1994 10 Jan. 123(2):36-38.
8. As noted in one commentary, problems arose in efforts to recruit women for trials designed to assess the benefit of HDC/ABMT in the treatment of metastatic breast cancer. Because of the availability of HDC/ABMT outside of clinical trials, many women with metastatic breast cancer were not willing to accept the chance of being randomized to a control group in a trial designed to evaluate the effectiveness of HDC/ABMT. As a result, it took much longer than expected to obtain an adequate number of participants in these studies to resolve the uncertainty over the value of this technology. (Steinberg, Tunis, and Shapiro 1995: 150. See also Kolata 1995; Kolata and Eichenwald 1999: A-1.)
9. Of five studies released in 1999, four indicated that high-dose chemotherapy with bone marrow transplant was no better for breast cancer than conventional chemotherapy. A fifth study, done in South Africa, suggested some benefit. However, several months later, as scientists looked at this study more closely in an effort to replicate its results, the principal investigator admitted to having falsified some of the data "out of a foolish desire to make the presentation more acceptable" to the scientific meeting sponsored by the American Society of Clinical Oncology. (Weiss et al. 2000: 1003. See also Antman, Heitjan, and Hortobagyi 1999: 1701-1703; Gradishar 1999: 1378-1380; Rowlings et al. 1999: 1335-1343; Horton 2000: 942-943; Bergh 2000: 944-945).
Another study completed even more recently reached the same conclusion, namely, that bone marrow transplant offers no advantage over conventional chemotherapy. (See Stadtmauer et al. 2000: 1069-1076; Lippman 2000: 1119-1120.)
Interestingly, the wide availability of insurance coverage effectively precluded completion of scientific trials on ABMT for ovarian cancer. (See Kolata and Eichenwald 1999: A-1.)
10. Associated Press 2000: A-5. It might also be noted that during the heyday of using ABMT for breast cancer, many hospitals and physicians made enormous sums of money from the treatment. See Kolata and Eichenwald 1999: A-1.
11. The issue is actually more complicated than this, because it is not simply a matter of courts permitting bad science (or no science) to dictate medical standards. As discussed below, the matter is complicated by health plan contracts that actually look for physician acceptance as their criterion of medical necessity.
12. In a fairly well-known phenomenon of which the ABMT episode is but one example, in "judge-made insurance" courts require payers and sometimes even physicians to provide even very high-cost, relatively unproven technologies to patients who otherwise lack much hope for survival. (See Abraham 1981: 1155; Ferguson, Dubinsky, and Kirsch 1993: 2116; Kalb 1990: 1118; Morreim 1995: 251.)
In another context, it has been observed that courts ruling on managed care cases can be remarkably ill-informed about the economic arrangements and financial structures of the health plans whose actions they judge, with the result that important decisions are sometimes based on badly misinformed factual premises. (See Morreim 2000a: 699-728.)
13. For examples of guidelines, see the following Web sites: www.clinicalevidence.org (Clinical Evidence, published by British Medical Journal); www.guideline.gov (National Guideline Clearinghouse™); www.ama-assn.org (Clinical Practice Guidelines Directory); 1ww.com (U.S. Preventive Services Task Force Guide to Clinical Preventive Services).
15. In Shannon v. McNulty (718 A.2d. 828, 835 [Pa. Sup. Ct. 1998]), a Pennsylvania superior court noted that "these decisions may, among others, limit the length of hospital stays, restrict the use of specialists, prohibit or limit post-hospital care, restrict access to therapy, or prevent rendering of emergency room care. While all of these efforts are for the laudatory purpose of containing health care costs, when decisions are made to limit a subscriber's access to treatment, that decision must pass the test of medical reasonableness" (emphasis added). See also Crum v. Health Alliance Midwest, Inc., 47 F. Supp. 2d 1013 (C.D. Ill. 1999); Wickline v. State of California, 192 Cal. App. 3d 1630, 1645 (1987);
16. Recently two physicians filed suit against Milliman and Robertson. "Two pediatricians, whom M&R cites as the authors of its pediatric hospitalization guidelines, allege not only that they did not write the guides, but that the guides are dangerous and inappropriate for pediatric care." (See Page 2000: 1, 29, 35.)
17. See, e.g., McGraw vs. Prudential Insurance Co. of America, 137 F.3d 1253, 1256 (10th Cir. 1998) (citing Prudential's definition of medical necessity partly in terms of being "recognized throughout the Doctor's profession as safe and effective"); Healthcare America Plans, Inc., v. Bossemeyer, 953 F. Supp. 1176, 1188 (D. Kan. 1996), affirmed without opinion, 166 F.3d 347 (10th Cir. 1998) (defining "experimental" treatments as those "not generally accepted by the medical community"); Miller v. Whitburn, 10 F.3d 1315, 1320 (7th Cir. 1993) (discussing Medicare's definition of medical necessity in terms of "whether the service has come to be generally accepted by the professional medical community as an effective and proven treatment for the condition for which it is being used" [citing Rush v. Parham, 625 F.2d 1150 (5th Cir. 1980)]); Horvath 1999: 455 (citing the National Blue Cross and Blue Shield Association's definition of "experimental and investigational treatment" partly in terms of whether it is generally "recognized by the medical profession as tested and accepted medical practice").
18. For an extended discussion of the ways in which the concept of medical necessity figures in health care contracts, and the problems inherent in that approach, see Morreim 1999: 1010-1025; Morreim 2000b: 144-158. Even in Medicare plans that ostensibly provide the same benefits package to everyone, a recent study showed that Medicare payment for a chest Xray was 451 times more likely to be denied in Illinois than in South Carolina, and payment for real-time echocardiography was nearly one hundred times more likely to be denied by Transamerica Occidental than by Blue Shield of California (Pretzer 1995: 92-93. See also Gleason 1995: 1483; Mariner 1995: 236-246.)
20. For a detailed discussion of this point with numerous cites, see Morreim 1999: 989-998.
21. Indeed, the most pristine kind of science—the randomized, double-blind, controlled trial—can sometimes be least applicable in the clinical setting. In order to test strictly for the effects of the specific drug or procedure under investigation, scientific study design must be restricted to patients fitting a narrow set of eligibility criteria. Typically they must suffer exclusively from the particular disease whose treatment is being studied, with a minimum of other diseases or medications that could confound the results. Once the study is complete, however, its results are applied in clinical practice to all those complex patients who would never have been eligible to be test subjects in the study.
One result of this misfit between the original study's narrowly identified participants, and the ordinary folk who later use the drug, is that sometimes even well-researched new drugs and procedures must be quickly withdrawn from the market, as they suddenly produce undesirable results and side effects that were not seen during the research period. Between September 1997 and September 1998, five FDA-approved drugs were removed from the market because of unexpected side effects or interactions with other drugs. (Friedman et al. 1999: 1729.)