Your browser doesn't support JavaScript. Please upgrade to a modern browser or enable JavaScript in your existing browser.
Skip Navigation U.S. Department of Health and Human Services
Agency for Healthcare Research Quality
Archive print banner

This information is for reference purposes only. It was current when produced and may now be outdated. Archive material is no longer maintained, and some links may not work. Persons with disabilities having difficulty accessing this information should contact us at: Let us know the nature of the problem, the Web address of what you want, and your contact information.

Please go to for current information.

Quality Interagency Coordination (QuIC) Task Force
Return to QuIC Home
About QuIC
Steering Group
Press Releases
Related Links
Site Map
Written Statement


National Summit on Medical Errors and Patient Safety Research

Panel 2: Broad-based Systems Approaches

Testimony of Robert Crane, Senior Vice President, Kaiser Foundation Health Plan, Inc. and Kaiser Foundation Hospitals and Director, Kaiser Permanente Institute for Health Policy

The first National Summit on Medical Errors and Patient Safety Research was held on September 11, 2000, in Washington, DC. Sponsored by the Quality Interagency Coordination Task Force (QuIC), the Summitís goal was to review the information needs of individuals involved in reducing medical errors and improving patient safety. More importantly, the summit set a coordinated and usable research agenda for the future to answer these identified needs.

Individuals were selected by the Agency for Healthcare Research and Quality (AHRQ) to testify at the summit as members of the witness panels. Each submitted written statements for the record before the event, documenting key issues that they confront with regard to patient safety as well as questions to be researched. Other applicants were invited to submit written statements.

Disclaimer and Copyright Statements

1. Background

My name is Robert Crane. I am a Senior Vice President with Kaiser Foundation Health Plan, Inc. and Kaiser Foundation Hospitals and Director of the Kaiser Permanente Institute for Health Policy. Part of my responsibilities includes oversight of the Kaiser Foundation Research Institute. I appreciate the opportunity to present our views on research priorities in the area of patient safety and medical error reduction. These views are set forth in Section 4 of this testimony. Kaiser Permanente is a non-profit, integrated health care delivery system serving 8 million members in 11 states and the District of Columbia. We are the oldest and largest not-for-profit health maintenance organization in America, with over 10,000 physicians, representing all specialties and over 90,000 non-physician employees who work in health care delivery, technical, administrative, and clerical positions.

The Kaiser Permanente Medical Care Program ("Kaiser Permanente") is a functional partnership between Kaiser Foundation Health Plan, Inc. and Kaiser Foundation Hospitals and The Permanente Medical Groups. This organizational structure represents a cooperative approach by the professions of medicine and management, and assures physicians and managers an opportunity to present their views on all major policy decisions. Our organizational structure allows us to involve and influence providers in ways that work to the advantage of the organization, our providers, and our members. Because we contract with the Permanente Medical Groups—multi-specialty group practices caring almost exclusively for Health Plan members—to deliver care, we are able to monitor outcomes and affect change on a broad scale.

2. Kaiser Permanenteís commitment to patient safety and medical error reduction

Kaiser Permanente believes that quality health care and patient safety are closely linked. Kaiser Permanente is an integrated health care delivery system whose approach to quality addresses providers, patients and systems. Patient safety is an important component of our quality assurance and improvement program. We believe that Kaiser Permanente can deliver high quality, safe, reliable care to its members by: (1) selecting the most effective providers, (2) supporting their professional development and training, (3) creating health care teams, (4) collecting and sharing quality-related performance information and effective practices, (5) encouraging the use of clinical guidelines and evidence-based medicine, (6) supporting patient education, and (7) investing in supportive technology. We also recognize that quality and safety improvement is a continuous process.

Over the past several years, Kaiser Permanente has been actively involved in leading national efforts to improve patient safety. We are a founding member of the National Patient Safety Foundation, and a Permanente physician is currently on its Board. David Lawrence, MD, the CEO of Kaiser Foundation Health Plans, Inc. and Kaiser Foundation Hospitals, is a member of the IOM Committee that authored the recent report To Err Is Human: Building a Safer Health System. Moreover, to begin to examine and answer the key questions concerning medical error reporting, and to define design principles for reporting, Kaiser Permanente has sponsored two roundtable discussions involving national experts on the topic of "Reporting as a Means To Improve Patient Safety".

Since the majority of medical errors do not result from malicious intent or incompetence but from the way health care is organized, Kaiser Permanente believes that its group practice and integrated care delivery systems are key ingredients of quality and patient safety improvement. In addition, we have undertaken several national initiatives to improve patient safety:

National Clinical Information Systems Project—We are currently developing a national clinical information system that will enable our physicians to easily access our membersí integrated medical records. This system gives us additional opportunities to reduce medical errors and improve patient safety.

Labor Management Partnership—Kaiser Permanente and a coalition of AFL-CIO unions representing 60,000 of our unionized health care workers have joined forces to promote patient safety practices within Kaiser Permanente's medical facilities nationwide. Kaiser Permanente and its unions are developing strategies to more fully engage the union-represented workforce in efforts to develop and enhance current and future patient safety initiatives and to provide them the protection they need to fully participate. This partnership will allow us to better address medical errors.

Drug and Disease Management Programs—Our physicians and pharmacists are continually working to develop and implement comprehensive drug and disease management programs to ensure timely and appropriate pharmaceutical use and to decrease adverse drug reactions. We have established a Care Management Institute dedicated to the synthesis of evidence, definition of successful practices, and the support of implementation of those practices.

Significant Event/Root Cause Analysis—We have implemented a significant event reporting and root cause analysis process that systematically documents and assesses significant events (unexpected deaths and other patient safety issues). Our medical staff is trained in root cause analysis, which is a rigorous analytical process that identifies the factors that contribute to the significant event. Findings and corrective actions from each significant event are aggregated in a database. Data from individual events are compared, and systems issues are identified and reported to the highest-level quality committees for prioritization and direction. These learnings are shared across our Program to prevent similar significant events in the future.

3. Kaiser Permanente Research Centers

The research focus of this Summit is encouraging. For over three decades, the Kaiser Permanente Medical Care Program has conducted formal research dedicated to improving the quality of medical care and health care delivery. Kaiser Permanente research has been widely published and over the years has played a significant role in shaping national health policy. While research is carried out in a variety of settings throughout our Program, it is concentrated in three principal centers:

The Division of Research

Over the past 35 years The Permanente Medical Groupís Division of Research in Oakland, California, has made numerous important contributions to medical, epidemiological, and health services research. At a glance:

  • The Division of Research conducts long-term studies on a base of 3.0 million Northern California Kaiser Permanente members; over 1000 published studies.
  • 1999 funding was $21.5 million; staff of 350 with 40 MD and Ph.D. investigators presently conducting approximately 200 research projects. Funded primarily by competitive federal grants and contracts.
  • Close collaborative ties include University of California at Berkeley, San Francisco, and Davis; Stanford University; and many other U.S. and international academic centers.

The Center for Health Research

The Center for Health Research in Portland, Oregon and Honolulu, Hawaii was founded in 1964 to serve the public interest through research, demonstrations, education, and community service. At a glance:

  • Study base: 445,000 members of the Northwest Region; conducts clinical, epidemiological, health behavior, and health services research.
  • Permanent staff of 25 investigators (MDs and PhDs), technical staff of more than 235.
  • In 1999, expanded research operations to the Hawaii Region, creating CHR/Hawaii. Collaborates with universities throughout the Pacific Northwest and in Hawaii; affiliated with Oregon Health Sciences University and Portland State University.
  • In FY 2000 funding is $20 million; currently conducting more than 120 research projects. Funded primarily by competitive federal grants and contracts.

Research and Evaluation Department

The Research and Evaluation Department was established in 1987 in Southern California to perform clinical epidemiological and health services research. At a glance:

  • Research base: Kaiser Permanente—Southern California serves an ethnically and socioeconomically diverse population of 3 million residents who receive care in 11 medical centers and over 1000 medical office buildings.
  • The Research and Evaluation Department had over 4000 active protocols in 1999. Ninety percent of funding for specific research projects comes from researchers in federal and foundation grants and from pharmaceutical companies.
  • Researchers in Kaiser Permanente-Southern California authored or co-authored more than 190 papers that were published in scientific journals in 1998-1999.

4. Research Suggestions

In response to AHRQís solicitation of patient safety research priorities, Kaiser Permanente has identified five areas of potential research. These areas include: Understanding Outcomes and Identification of Successful Practices; Human Factors and Safety System Engineering; Use of Technology; Reporting of Medical Errors; and Culture and Environment.

A. Understanding Outcomes and Identification of Successful Practices

Understanding outcomes and identification of successful practices is integral to the process of improving quality of care and ensuring that health care delivery systems become safer. We believe that there is a significant opportunity to reduce adverse events through the synthesis of knowledge of the best clinical practices and the subsequent creation, implementation, and evaluation of successful practices. There are a number of quality of care and patient outcome studies that we believe can contribute to advances in the reduction of adverse events in medicine.

Market-based Initiatives

A group of national employers, The Leapfrog Group, has recommended, in an effort to increase patient safety, that elective hospitalizations for selected procedures in urban areas be provided in hospitals that meet favorable hospital volume characteristics and demonstrate mortality rates below regional norms.


Favorable Hospital Volume Characteristic

Coronary artery bypass

Volume > 500/year

Coronary angioplasty

Volume > 400/year

Abdominal aortic aneurysm repair

Volume > 30/year

Carotid endarterectomy

Volume > 50/year

Esophageal cancer surgery

Volume > 7/year

Delivery with expected birthweight < 1500 grams or gestational age < 32 weeks

Regional neonatal ICU with average daily census > 15

Delivery with pre-natal diagnosis of major congenital anomalies.

Regional neonatal ICU with average daily census > 15

(Source: The Leapfrog Group, Draft Patient Safety Standards, January 2000)

Select for Text Version.

This is an important development because it represents the most focused effort to date in which multiple purchasers are uniting to influence the practice of medicine. A general evaluation of the effectiveness of this effort or other examples of purchaser groups setting standards and affecting patterns of care will be valuable. It would be useful to quantify the degree to which the outcomes for the targeted conditions and procedures improve as a result of the initiative. Consideration should also be given to studying the hypothesis that the difference in performance between high and low-volume hospitals may dissipate over time without the initiative, as the lower-volume institutions adopt the successful practices of the high-volume facilities.

An effort should also be made to study whether there are adverse consequences as a result of the proposed initiative (e.g., deterioration of quality and safety in lower-volume hospitals that must care for patients with the targeted conditions or conditions that require the skills of surgeons or technical staff who now practice only in high-volume institutions). This study or a separate effort could also identify and evaluate the processes in the higher-volume/lower-mortality hospitals that generate better outcomes. Successful practices should then be promoted.

Using Large Integrated Data Systems to Identify Improvement Opportunities

Data systems in health care organizations are maturing rapidly. For example, at Kaiser Permanente, we are currently developing a national clinical information system that will enable our physicians to instantly access their patientsí entire integrated electronic medical records. This project should play a significant role in reducing medical errors and improving patient safety. Opportunities currently exist to use large integrated data systems (such as those currently in place in Kaiser Permanente) to find and study antecedent conditions of errors and high-risk situations. This work would build on the capabilities of existing data systems in place, particularly in large organized delivery systems, and would employ the familiar QI model of monitoring or profiling to identify outcomes or combinations of outcomes that would signal a greater than usual rate of errors or adverse events in a specific practice or a facilityís population. Other outlier facilities would then be examined in greater detail to determine whether system or human error explains their unfavorable variation.

Examples of areas for the study of antecedent conditions of errors and high risk situations include prolonged lengths of hospital stay for routine admissions such as appendectomy, cholecystectomy, hysterectomy, c-section, CABG; use of transfusions in situations where transfusions are not expected; infection rates; occurrence of other secondary discharge diagnoses that could indicate safety problems. Errors may simply be too rare to create detectable differences in rates that also have other determinants. However, if this approach were feasible, it would have several advantages over research based on direct reporting of adverse events. Studies could begin today, as the data systems are available. And the methodology may present fewer concerns of "discoverability" as the data create less legal liability for an institution, at least at first glance. In addition, this approach has much less susceptibility to variability in reporting, so that differences across facilities and over time are more believable.

Evaluation of Successful Practices

We believe that an examination of patient care outcomes should be undertaken to identify successful practices in error reduction and quality improvement. A variety of initiatives are now underway across the Kaiser Permanente Medical Care Program. These include numerous efforts to improve prescription practices, as well as organizational change efforts. There will be opportunities to evaluate these initiatives using either randomized or quasi-experimental designs.

B. Human Factors and Safety Systems Engineering

Others at this Summit have significant direct experience in this area. However, human factors and safety systems engineering have been employed by some health care organizations to collect and analyze safety-related information in order to improve care processes and provide safety-related training for medical professionals. There are significant opportunities to expand this research area in the future.

Documentation of Human Factors Used to Improve Patient Safety

An example of useful studies in this area is the evaluation of the effectiveness of using multiple levels of safety systems and procedures to help pharmacists deal with look-alike and sound-alike drugs. Questions include: What are the human factors leading to error? How can human factors knowledge create more effective prevention of errors? How effective are implemented changes? A study of the use of human factors as part of reengineering specific processes would be useful in advancing the health care systems knowledge about approaches in this area. A pre-post evaluation or an inter-organizational comparative analysis is well suited to this type of study.

Another example is the examination of the inpatient discharge processes that help to minimize post-discharge complications among high-risk groups, including the elderly. The circumstances this study would seek to minimize include medication non-compliance, functional decline, and re-hospitalization. Human factors experience would be applied at various patient levels to improve outcomes. This work would consider the psychological factors involved in patient compliance after inpatient discharge and its impact on safety. Research would then measure outcomes compared to situations where the intervention was not used.

Evaluation of Alternative Learning and Change Approaches

Despite a variety of initiatives to eliminate errors in operating rooms, emergency rooms, and ICUs, preventable adverse events persist in these settings. An evaluation of the impact of training and team factors on safety in the settings mentioned above is another area of research that holds the potential for significant value. This work calls for an examination of alternative learning and change systems in organizations focused on the implementation and monitoring of initiatives designed to eliminate errors.

C. Use of Technology

Use of technology in medical care delivery holds great potential for improving patient safety, and to date numerous strategies have been offered (e.g., automated prescription order entry, electronic medical record, and clinical alert systems). Technology also creates opportunities for new types of errors.

Evaluation of New Technology to Improve Patient Safety

A rigorous evaluation of the cost-effectiveness of emerging technologies is needed to help organizations prioritize investments for quality and safety improvement. In Kaiser Permanente, automation in a number of locations helps prevent pharmaceutical errors from occurring. Systems can instantly provide physicians with all critical member information normally only available on paper charts. After a physician has written the patientís prescription, the prescription goes through a series of checks before any drugs are dispensed: medication allergies, drug to drug interaction, drug overlaps, therapeutic substitution, and accuracy in filling.

Increasingly, vendors are marketing a variety of innovative products with claims that they are a means to improve safety and reduce errors. Examples of these products include automated pharmacy dispensing systems, handheld electronic prescribing devices, electronic medical record and computer order entry, and bar code medication administration control systems. Given the variety of technological interventions available today to address patient safety, there will be great value in efforts to validate the efficacy of these technologies and to quantify their investment yield. It makes little sense for individual hospitals to duplicate such studies. Making good decisions about investments is of great importance given limited capital. For example, in California, hospitals face stringent seismic standards to protect patients and staff that will require many institutions to make huge investments in facility construction and retrofitting. This heightens the need for a means for these institutions to prioritize the investment of their remaining capital, some of which must be available for other patient safety improvements. We believe that there are many opportunities to use existing data to evaluate these technologies. AHRQ can play an important role in supporting the development and evaluation of business cases around the investment in certain classes of technology to improve patient safety.

The Potential Untoward Impact of Technology

We must also acknowledge that technology designed to promote safety has the potential to amplify negative consequences. Concerns have been raised that the new sense of security created by emerging technology may lead health care professions to become lax in their use of established protocols for errors avoidance. We see value in studying how new technological innovations as well as more established non-technical error reductions strategies interact. There is also value in aggregating reports of adverse incidences related to the use of technological interventions that are designed to improve safety. We also see value in the exploration of the effectiveness of machine-human interface in creating safer or error-proof medical care processes.

D. Reporting of Medical Errors

Reporting of medical errors and near misses was broadly discussed in the IOM report, To Err Is Human, as a means to increase accountability and improve patient safety in medical care. Through our involvement with efforts to explore the design elements of a national medical event reporting strategy, we have identified the need for research in this area. We support the evaluation of existing reporting systems to determine the characteristics and features that have a positive impact on patient safety.

Evaluation of State Mandatory Reporting Systems

A number of states currently require mandatory reporting of adverse events from general and acute care hospitals. The focus, structure and operation of these reporting systems differ significantly. Given the IOMís recommendation to build a mandatory, national system of state-based reporting, an evaluation of these existing state systems is particularly important to determine their impact on patient safety improvement. For this analysis, five to six markers (e.g., unanticipated death, suicide of a patient, wrong site surgery) could be used to compare the rate of incidence in states with and without mandatory reporting systems. An alternative or complementary approach might be to compare the responses to survey questions regarding patient safety orientation among providers in states that mandate reporting versus providers in states that do not.

The Factors Impacting Reporting

We also see value in research to determine the factors that hinder reporting compliance at the facility level to identify errors that require system fixes. We expect a number of facilities to establish more robust internal error reporting systems as a result of the IOM work. An evaluation of the factors that underlie the success of these systems would be useful, particularly emphasizing the translation of causes of errors into system changes for safety improvement. In addition, a recent roundtable discussion sponsored by the Kaiser Permanente Institute for Health Policy, NASA's Aviation Safety Reporting System and the Quality Forum highlighted the desirability of evaluating existing voluntary error reporting systems that operate on a national or international basis as well as the pilots that develop, such as that undertaken by the Veterans Administration in conjunction with the Aviation Safety Reporting System.

E. Culture and Environment

Culture and environment in the health care industry are often described as limiting factors in efforts to discover system vulnerabilities and encourage collaboration among medical professionals to seek solutions. Last Fall, the IOM report highlighted the need for health care organizations to develop a culture of safety such that an organizationís care processes and workforce are focused on improving the reliability and safety of care for patients. More recently, a group of health care organizations developed a set of general principles for patient safety reporting system. At the top of the list of principles is the creation of a safe, non-punitive environment for the reporting of health care errors.

Over the past decade, health care organizations have made significant improvement in the assessment of physician and employee satisfaction with various aspects of the care delivery system and the work environment. We believe that there are key questions that should be incorporated into physician and health care employee survey instruments to identify the factors at the health care team and facility levels that are needed to transform the current culture of blame. Both cross-sectional surveys and before/after surveys with or without randomization to interventions would be useful. We also believe that AHRQ can take the lead in sponsorship of an effort to develop a standardized set of patient safety oriented questions or a survey instrument for this purpose, which will allow comparability across institutions. In carrying out this work, it will be important to examine culture from at least three perspectives: (1) organizational culture that makes it safe to identify errors, (2) the related issue of an organizationís willingness to use data to make change, and (3) professional culture that influences willingness to acknowledge errors and share information.

5. Research Approach

Combinations of qualitative with quantitative approaches would be useful for much of the work that we have suggested. Most of the investigations by their nature will require looking across institutions and aggregations of data. We see a new opportunity for collaboration across several disciplines in research to improve patient safety—involving researchers, quality experts, and the operational stakeholders involved in medical care delivery. These parties should partner throughout the research process, from study design to the interpretation of results and identification of action. Achieving this partnership will require assurance of confidentiality, as reviewed below.

6. Confidentiality Protection

Research into medical errors and adverse events is a means to build a broad base of knowledge from which other health care providers across the nation could draw to improve safety within their institutions. Typically, confidentiality protection for self-examination by health care institutions occurs in the context of quality assurance and peer review activities that are conducted for internal use. Quality assurance and peer review activities are not typically considered research that is intended to develop or contribute to generalizable knowledge; the results of these studies are held privately and are rarely shared outside of the institution. Yet this information holds much potential for general patient safety improvement. There is a legitimate concern among health care institutions that research into patient safety concerns within an institution should be granted immunity from discovery prior to sharing of findings. For a number of studies in this area, the absence of assurance of confidentiality and/or protection from discovery in our litigious society will be a serious impediment to conducting some of the most useful research. Establishing a prohibition against subpoenaing researchers conducting patient safety research or related approaches to discovery is also important. In the interest of promoting learning from adverse events and errors, there is a need to extend protections to the institutions that sponsor and conduct such research.

We understand that researchers can now apply to the Department of Health and Human Services for a Certificate of Confidentiality, which is designed to protect the privacy of individuals who participate in research. Researchers who are granted a Certificate of Confidentiality may not be compelled to disclose information through subpoena about research subjects in any Federal, State, or local civil, criminal, or other proceeding, although there are exceptions (i.e., audit, FDA, subject request). But the scope of the protection granted by the Certificate of Confidentiality does not necessarily extend to the organization that sponsors the research. A Certificate of Confidentiality is only granted for research that is deemed to be of a sensitive nature. For example, genetic information is a specific category that is described as sensitive in federal documents. We question whether the current regulations are elastic enough to extend protections to institutions conducting research on adverse events and errors with the intent of sharing results with the greater medical community.

We recommend that AHRQ pull together a forum of interest groups with the purpose of designing appropriate protections and assurances that federally sponsored and other research designed to understand medical errors and improve patient safety will be protected. Federal legislation may be needed to implement such protections or to increase the confidence of institutions and researchers that protections will be effective. With appropriate confidentiality protections in place, a variety of studies could be conducted, such as those related to institution-based root cause analyses of adverse events and other quality improvement studies related to selected elective hospitalizations to determine the factors that contribute to better outcomes. This cross-institutional analysis will be particularly useful in identifying trends and patterns.

7. Conclusion

As outlined above, we see tremendous opportunity for the improvement of patient safety through research in at least five broad areas: Understanding Outcomes and Identification of Best Practices; Human Factors and Safety System Engineering; Use of Technology; Reporting of Medical Errors; and Culture and Environment. We recognize that these five areas are interrelated, which provides an opportunity to integrate research areas of focus into broader studies.

Research that will have the greatest impact is that which can be shared and or conducted across facilities, raising liability and discoverability concerns. As a priority for AHRQ, we recommend that you work for a clear solution to this issue so that we can maximize the learning and improvement that can come from research on medical errors and patient safety.

Current as of September 2000

Return to Testimony Directory
National Summit on Medical Errors and Patient Safety
QuIC Home Page
Department of Health and Human Services

The information on this page is archived and provided for reference purposes only.


AHRQ Advancing Excellence in Health Care