AHRQ's Patient Safety Initiative: Building Foundations, Reducing Risk: Chapter 1 (continued)

Chapter 1. AHRQ's Medical Error Reporting Demonstration Evaluation Projects (continued)

Reporting System Strengths and Need for Triangulation

The strengths and limitations of the three major types of reporting systems currently being evaluated in our demonstration projects are highlighted in the following tables.

Table 8. Spontaneous Active Reporting Systems

  • Involves health care providers.
  • Indicates change in organizational culture.
  • Includes known errors as well as no-harm events and near-misses.
  • Identifies potential risks and hazards.
  • Serves as basis for root cause analysis and determining factors contributing to error.
  • Cannot be used for establishing prevalence rates.
  • Under-reporting is significant.
  • Subject to reporter bias, hindsight bias, and organizational culture issues.
  • Staff commitment and involvement is inconsistent.
  • Quality of reports is uneven.

Two grantees found discharge data represent a major and often untapped resource for identifying potential indicators of patient harm.

Table 9. Administrative Data Systems

  • Uses data already available.
  • Captures large populations.
  • Examines State/regional/local trends.
  • Identifies significant problems for further investigation.
  • Establishes prevalence.
  • Distinguishing error from non-preventable harm is impossible.
  • Subject to coding variations.
  • Provides only indicators of harm.
  • Requires follow-up study.
  • Performing root cause analysis is impossible.
  • Identifies limited subset of error-associated injury.

Medical record data have been traditionally used as a means of identifying risk and hazards. With the advent of electronic medical records, two grantees found the use of triggers or indicators of harm show significant potential. A special form of medical record is that of the closed claim insurance files from malpractice cases.

Table 10. Medical Record Data Systems

  • Reports indicators of harm.
  • Runs in background (especially with electronic medical records).
  • Can be used proactively or retrospectively.
  • Works with known harms.
  • Inadequate for spotting rare or unusual events.
  • Insensitive to near misses.
  • Does not engage health care providers.
  • Cannot perform root cause analysis.

In addition, grantees concluded there is no single best type of reporting or surveillance system for detecting risk, hazards, and harm to patients associated with or caused by the delivery of their health care. The concept of truth or exact position being determined by multiple sources of information (as in navigation) is a lesson that might well be applied to medical error reporting. Being able to link data from different types of reporting and surveillance systems can be valuable.

Some reporting demonstration evaluation grantees are using multiple sources of data as part of their reporting and are demonstrating the importance of triangulating medical error data. The use of discharge data is proving to be an effective method of identifying potential risk areas for further study. The use of closed claims data is proving to be an important source of data for examining a variety of medical error types. New York State is finding that when data are protected, hospitals reflect an increase in the number of medical errors reported. They are also demonstrating the utility of combing active reporting from hospitals with that of using the discharge data for comparison (e.g., comparing findings from the SPARCS data to that reported through the New York Patient Occurrence Reporting and Tracking System).

Observation 2. As medical error reporting improves, error detection rates increase while error severity eventually decreases.

Earlier studies have noted that error detection is the first step in error management. The IOM report states that fear of repercussions (e.g., liability, loss of job, litigation) has resulted in substantial under-reporting of errors and avoidable harms. Thus, 13 grantees found an early indication of the success of efforts to address medical errors will be an increased rate of errors. It is essential that this apparent increase be understood as a positive step. From an organizational point of view, it is important that the error detection rate be high because errors that go undetected cannot be prevented and harm cannot be mitigated. Thus, one of the goals in patient safety management is to increase detection and reporting thereby decreasing the potential risk of harm to patients. These 13 grantees argue organizations should aspire to high levels of error detection and reporting. However, these efforts must be balanced by the practical costs of conducting the reporting, i.e. costs of changes in organizational infrastructure to improve error detection. Above all, it is important to do no harm.

What we know from other industries which have made an effort to improve safety is that the number of errors reported will go up but the severity of the reported errors will eventually go down. Thus, over time, the percentage of errors detected and reported should remain high while the severity of those reported errors is expected to decline as an indicator of successful error management.

Safety experts such as Dr. James Reason argue that the goal of safety should not be a reduction in the number of errors detected and reported, but rather a mitigation, reduction, or elimination of the risks and hazards leading to error and patient harm.4 AHRQ's measure of success will not be the reduction of the number of errors reported, but rather the prevention of patient injury associated with health care. In the early stages of AHRQ's Patient Safety Initiative, success should be measured by an increase in the detection of errors, risks, and hazards—not a reduction in reported errors. Increases in error reporting are expected to lead to a reduction or elimination of the risks and hazards leading to patient injury.

In the last year-and-a-half, AHRQ has learned important lessons about the climate and incentives necessary for medical error reporting to be successful. Where there is strong leadership support for reporting and a sense of trust, reporting notably increases. For example, preliminary information from the Georgia Hospital Association and Washington University error reporting demonstration evaluation projects clearly show an increase in the reporting of errors in response to supportive leadership. It is too early in the process to expect decreases in the severity of the errors reported.

Observation 3. Despite July 2001 patient safety standards from JCAHO that require hospitals to disclose to patients all unexpected outcomes of care, there appears to be great variability in the degree and process of error disclosure. While providers are concerned that greater disclosure may increase the likelihood of malpractice claims, in fact, the opposite appears to be true. Prompt and full disclosure has been shown to reduce malpractice claims and settlements.

There has been considerable public concern that health care organizations are reluctant to provide patients with information on occasions where there has been harm due to or associated with the delivery of health care. Prompt and full disclosure has been shown to reduce malpractice claims and settlements.5 Patient safety standards from JCAHO that require hospitals to disclose to patients all unexpected outcomes of care took effect July 2001. Nonetheless, there appears to be great variability in how medical errors are disclosed.

The Agency also sought applications for the medical error reporting demonstration evaluation that would test various methods of informing patients and their family members when an injury due to or associated with the delivery of health care occurred.

Seven of the 16 projects are operating in environments in which providers are required to disclose errors to patients or their families. In several multi-site projects (Georgia Hospital Association, Columbia University, and Washington University), providers in some, but not all of the sites are required to disclose information to patients or their families.

Table 11. Disclosure of Errors to Families Required

InstitutionInvestigatorDisclosure to Families
Columbia UniversityKaplanYes
Johns Hopkins UniversityPronovostSome
Georgia Hospital AssociationThorpeYes
New York State Health DepartmentOstenYes
University of Colorado Health CenterPaceSome
University of Mississippi Medical CenterBrownYes
Washington UniversityFraserYes

While in general it is too early in the life cycle of the research projects for results to be available on how to address medical errors with patients and/or their family members, two of the grantees have undertaken specific studies on the issue which have recently been published in the medical literature.

Studdert and colleagues participated in the hospital disclosure practices study to learn more about how hospitals disclose information to patients. In early 2002, a survey of risk managers at a nationally representative sample of hospitals showed that the majority reported that their hospital's practice was to disclose harm at least some of the time. However, only one-third of the hospitals had board-approved policies in place. More than half of the survey respondents reported that they would always disclose a death or serious injury, but when presented with clinical scenarios, respondents were much less likely to disclose preventable harms than to disclose non-preventable harms of comparable severity. Reluctance to disclose preventable harms was twice as likely to occur at hospitals having major concerns about the malpractice implications of disclosure.6

In an AHRQ-supported project at the University of Washington School of Medicine, Gallagher and colleagues conducted focus groups to determine patients' and physicians' attitudes about medical error disclosure. The researchers found that both patients and physicians are not satisfied with how errors are presently disclosed. Patients wanted disclosure of all harmful errors and sought information about what happened, why the error happened, how the error's consequences will be mitigated, and how recurrences would be prevented. Physicians agreed that harmful errors should be disclosed but "choose their words carefully" when telling patients about errors.7

Although physicians disclosed the adverse event, they often avoided stating that an error occurred, why the error happened, or how recurrences would be prevented. Patients also desired emotional support from physicians following errors, including an apology. However, physicians worried that an apology might create legal liability. Physicians were also upset when errors happen but were unsure where to seek emotional support. They concluded that physicians may not be providing the information or emotional support that patients seek following harmful medical errors. Grantees determined that physicians should strive to meet patients' desires for an apology and for information on the nature, cause, and prevention of errors. Organizations should also address the emotional needs of practitioners who are involved in medical errors. This is consistent with the fact that JCAHO and some, but not all, State laws require that patients be informed about errors in their care.7

Observation 4. The ability to identify patient safety problems would be enhanced by availability of data guidelines and standards, i.e., common taxonomy and standardized data definitions and data elements in reporting systems.

The potential power to identify patient safety problems would be enhanced by the availability of data guidelines and standards (i.e., common taxonomy and standardized data definitions) as noted by several AHRQ grantees. To meet the need for standards AHRQ asked the IOM to develop recommendations for patient safety data standards. The IOM established the Committee on Data Standards for Patient Safety. The committee issued its report, Patient Safety: Achieving a New Standard of Care, on November 20, 2003. These standards will be implemented as appropriate by the Patient Safety Task Force and AHRQ.

Observation 5. Alignment of hospital and health care organization human resource policies and procedures with the emerging "just culture," which reconciles professional accountability and the need to create a safe environment to report medical errors, is a task for the hospital or organizational leadership.

Organizational culture varies considerably and directly affects organizations' and leaders' willingness to embrace changes necessary to improve patient safety. While issues of blame-free reporting have been an important part of the discussion of patient safety, balancing the need for professional accountability and the need for open communication about risks, hazards, and errors is a challenge that was shared by the grantees during their first 18 months.

James Reason, in his book Managing the Risk of the Organizational Accident,4 was the first to coin the term "just culture" which bridges the gap between reasonable accountability and no-fault reporting. David Marx has expanded the concept further and provided guidance for health care organizations. The leadership of hospitals and health care organizations should examine their human resource policies and procedures to ensure that they are aligned with the emerging "just culture," which reconciles professional accountability and the need to create a safe environment to report medical errors. Health care organizations and their leaders may need assistance and guidance in aligning their policies and procedures in a "just culture" environment.

Observation 6. Legal protection of reported data is critical if medical error reporting is to be truly effective in improving patient safety. Effective reporting requires encouragement of health professionals to report medical errors and other threats to patient safety, and a prevailing national norm of legal protection of all reported patient safety data.

Legal protection of both reporters and reported data remains a critical issue in medical error reporting as noted by some of AHRQ's patient safety grantees. For medical error reporting to be truly effective in improving patient safety, health professionals must be encouraged to report medical errors and other threats to patient safety. However, several of the grantees found that many organizations involved in their projects continue to have personnel policies that are incompatible with open and free reporting of events. To move to a more "just culture" as described above, significant changes in policies will be required. Several grantees suggest that legal protection of voluntarily reported patient safety data, particularly the no harm and near miss events, should become a prevailing national norm. Pending legislation in both houses of Congress is designed to address this critical issue.

Observation 7. Medical error reporting systems may be most effective if they are easy to use, but not so simple that the information reported is of limited value to the institution. The balance between the information required of the reporter and the system operator who investigates the reported events is critical to overall success.

An important lesson being learned in medical error reporting is that the reporting system must be easy to use. Our grantees are finding that a user-friendly reporting system enhances the number of reports that are received. However there is also a risk in making reporting so simple that the information reported is of limited value to the institution. If the information reported cannot be easily converted into usable data then the reports submitted are a waste of time, both for those who report and those who would use the data. The balance between the information required of the reporter and the system operator who investigates the reported events is critical to overall success. Considerable attention is being paid to this aspect of report design in the development of the Patient Safety Task Force Patient Safety Network project, one of the major activities of the Patient Safety Task Force. This project is designed to integrate the reporting systems of the DHHS including:

  • FDA's Adverse Event Reporting Systems for Drugs.
  • Manufacturer's Device Reporting System for the Center for Devices and Radiological Health (CDRH), FDA.
  • Blood Products Deviation Reporting for the Center for Biologics Evaluation and Research (CBER), FDA
  • An FDA program shared with the Centers for Disease Control (CDC), the Vaccine Event Reporting System.

An additional program of the CDC also is included: the National Healthcare Safety Network System for infections, health care worker safety, and end stage renal disease.

Observation 8. Observations suggest that analyzing the root causes of medical errors is a high priority in improving patient safety. Many of the root causes or errors are not case-, organization-, or domain-specific, but follow common patterns.

Lessons learned from our grantees and other research in event reporting has indicated that there are generic causes or contributing factors of events that follow common patterns. These have been broken down into human cognitive behaviors such as making slips (doing something you did not intend to do—errors of commission) and lapses (not doing something you were supposed to do—errors of omission) in well practiced routines. Another example of common human cognitive behavior is making a mistake by selecting the wrong rule of protocol for carrying out a procedure or selecting the correct rule but carrying it out incorrectly. Other contributing factors or causes include organizational factors including poor procedures, staffing and budgeting issues. Still other contributing factors include technical issues including inadequate design of equipment or facilities. When common classification of these factors is used, the causes in medical events are often quite similar to those in other industries outside of medicine. Kaplan and colleagues found that the causes of events in a transfusion service mapped closely with the cause and contributing factors of those occurring in a petrochemical plant in the Netherlands.8 Determining the most common root causes of medical errors with any degree of accuracy may be revealed with further progress of the demonstration evaluation projects.

Return to Contents



AHRQ has made significant progress in the area of medical error reporting demonstration evaluations over the past 18 months. The hard work of grantees, contractors, and other agencies in the DHHS is paying off. Nonetheless, we still have a lot more to learn from the activities of our reporting demonstration evaluation grants. These projects have begun to collect significant quantities of data about medical errors and patient safety breaches as well as their causes and preventative strategies.

Over the months ahead we anticipate a much more detailed picture about medical errors and breaches in patient safety and the ways in which they can be prevented, eliminated, or mitigated to improve the safe delivery of health care. The results of our 16 medical error reporting demonstration evaluation projects and the ongoing work of the Patient Safety Task Force together provide a firm foundation for better reporting and use of information that will lead to improved patient safety.

Return to Contents
Proceed to Next Section

Page last reviewed December 2003
Internet Citation: AHRQ's Patient Safety Initiative: Building Foundations, Reducing Risk: Chapter 1 (continued): Chapter 1. AHRQ's Medical Error Reporting Demonstration Evaluation Projects (continued). December 2003. Agency for Healthcare Research and Quality, Rockville, MD. http://archive.ahrq.gov/research/findings/final-reports/pscongrpt/psini1b.html