Panel 3: Particular Systems Issues
Testimony of Mark E. Bruley, ECRI
The first National Summit on Medical Errors and Patient Safety Research was held on September 11, 2000, in Washington, DC. Sponsored by the Quality Interagency Coordination Task Force (QuIC), the Summit’s goal was to review the information needs of individuals involved in reducing medical errors and improving patient safety. More importantly, the summit set a coordinated and usable research agenda for the future to answer these identified needs.
Individuals were selected by the Agency for Healthcare Research and Quality (AHRQ) to testify at the summit as members of the witness panels. Each submitted written statements for the record before the event, documenting key issues that they confront with regard to patient safety as well as questions to be researched. Other applicants were invited to submit written statements.
Disclaimer and Copyright Statements
I am Mark Bruley, Vice President for Accident and Forensic Investigation
with ECRI, an independent, nonprofit health services research organization
devoted to the assessment, planning, procurement, and management of
health care technology. We have a special emphasis on medical devices
and maintain laboratories where this technology is evaluated in a fashion
similar to Consumer Reports. ECRI does not accept funding from the
medical device or pharmaceutical industries, and functions at arm's length
from them. The results of our research are used in the overwhelming
majority of the nation's larger- and medium-sized hospitals, and in many
healthcare institutions abroad. We are designated as an Evidence-based
Practice Center by AHRQ and as a Collaborating Center of the World Health
Organization. We have 250 employees, worldwide, and a large variety of
programs to ensure patient safety. My particular expertise derives from
investigating accidents and problems with medical devices in both the
hospital and laboratory settings over the past 25 years.
ECRI's free Medical Device Safety Reports web site
(http://www.mdsr.ecri.org) is a repository of medical device incident and
hazard information independently investigated by ECRI. In regard to
medical errors involving medical devices, ECRI's research draws on its
expertise in evaluating medical devices in bench-scale laboratory tests,
from formal, in-depth investigations of more than 2,000 adverse device
events in the field at the incident institution and/or at ECRI's laboratories,
and in investigating more than 25,000 medical device problem reports
submitted to ECRI since 1971 by healthcare institutions through our
international medical device problem reporting network.
As a biomedical engineer at ECRI, I have learned a great deal about the
causes of accidents and problems and have been able to analyze many of
the causes of these adverse events. However, I stress at the outset that the
firm research base that would meet the standards that are expected from
health services research does not exist related to medical errors involving
healthcare technology. Therefore, one of my key messages in discussing a
research agenda today is that we must develop that research base.
Specifically, we do not have a "denominator" or baseline from which to
launch truly meaningful studies. I will describe my approach to how we
might begin to construct a denominator in the course of this presentation.
The lack of good quantitative data and research has not, and should not,
stall the effort to improve equipment-related safety. The experiences I have
had and the programs my organization has developed, have made an
extremely strong contribution over the decades to improving patient
safety. Qualitative studies allow us to understand how many of these
accidents and adverse occurrences take place and to design corrective
measures. I am proud of my organization's work, but unfortunately, we are
likely to have a great deal more work to do in future. Let me give you an
example of the qualitative thinking that I think is helpful. The 1999 Institute
of Medicine (IOM) report on medical error cites an example of a free-flow
problem with intravenous infusion pumps in the operating room (39). The
example is highlighted throughout the text of the IOM report and is
specifically used to show corrective actions that could have been taken by
personnel and systems that could have been erected to avoid error. But
what is missing from that example's analysis, is something more
fundamental. The hospital should never have purchased an infusion
pumps that permitted free flow. Recommendations have long been
available from ECRI that would have guided an equipment acquisition
process that would never have permitted this type of accident to occur.
These omissions in the IOM report example point to the complex nature of
healthcare technology accident analysis and remediation and to the need
for medical device accidents to be addressed in a research agenda on
medical errors and patient safety.
Defining a Research Agenda
Medical device related errors may account for a minority of errors in
medicine, but they are, nonetheless, a critical component in the
development of a research agenda for improved patient safety. Because
their analysis for causation reveals much about the genesis of medical
error, user limitations, human factors, and system design, it is essential to
increase the attention given to the safe design, use, and management of
medical devices. The healthcare technology research agenda must
obviously extend beyond the medical error discussions and debates that,
so far, have generally limited their focus to the ubiquitous example of
"medication errors with infusion devices."
More research is needed to provide baseline or denominator data for
studying medical device-related errors as has been put done to great
positive effect already in, for example, anesthesia management (13, 14, 15,
28, 33, 34, 51, 57), hospital pharmacy and medication delivery (3, 5, 12, 31,
32, 38, 40, 59), and cardiopulmonary bypass perfusion (4, 18, 36, 43, 44).
Although equipment failures, per se, are rare in these areas, most of the
related medical errors occurring in these settings do involve the use, or
mis-use, of a medical device. The human-machine interface in the surgical
setting, and in particular in the practice of anesthesia, has been analogized
to that of pilot and airplane (45, 51). As in the aviation industry, it is
important to study these events to identify and examine correlations
between contributing factors before hypotheses about causation can be
formulated and tested. This is the first step in preventing medical errors
involving adverse device events and the patient injuries associated with
Virtually every aspect of medical care, diagnosis, treatment, and therapy
involves the use of multiple technologies, ranging from simple needles and
syringes to complex computer-based monitoring or diagnostic systems.
The healthcare setting is replete with medical devices, many of which have
caused or contributed to numerous patient deaths and injuries, and
countless potential injuries or "near misses." (See, in general, the
hundreds of thousands of Medical Device Reports collected by the U.S.
Food and Drug Administration, Rockville, MD; see also Health Devices
Alerts published by ECRI, Plymouth Meeting, PA.) Indeed, the public safety
concerns on medical devices have spurred a plethora of U.S. legislative
and regulatory requirements, including a mandatory reporting scheme for
medical device deaths and injuries (Sections 519(a), (b) of the Federal
Food, Drug, and Cosmetic Act, as amended, 21 U.S.C. 360i.(54)).
These issues are especially acute in technology intensive medical
specialty settings where one would expect accidents or adverse events
and the device-related research agenda should concentrate, naturally
enough, on these specialty areas. These areas include intensive care
medicine, emergency medicine, obstetrics and gynecology, general
surgery, cardiac catheterization, clinical laboratory, and respiratory
therapy. A research agenda focused on these technology-intensive medical
specialties will provide specialty-specific data of accident and error
causes, i.e., denominator data, that is necessary for undertaking
meaningful studies of patient safety. Broad-based research approaches
are not likely to be effective.
While most medical errors have multiple causes, there are three
"constants" in any such accident: the medical device or technology, the
procedure being performed, and the injury that results. It is in the areas
where these constants intersect (i.e., in the correlations between them) that
the causes of accidents are discovered and from which the
recommendations to prevent future incidents are derived. Within the
context of the high risk medical specialty settings, ECRI believes that
research, constructed on the narrower of the two 1999 Institute of Medicine
(IOM) report definitions related to medical errors, i.e., "accidents" and
"adverse events" (39), should be targeted on investigating these
relationships by addressing the following key questions related to
- To what extent do medical devices and related information systems
contribute to medical errors (including medication errors), especially
within the technology intensive medical specialties?
- What engineering controls, including those based on continuing
human factors research, can be employed to minimize the likelihood
of medical errors?
- Does the frequency and severity of device-related error justify
developing broader reporting and prevention interventions?
Constructing a research agenda on "medical error" or "patient safety"
related specifically to medical devices and these posed questions will
depend on the scope given to these terms. The 1999 IOM report does raise
some medical device related issues. It concentrates on preventing
"adverse events" and accidents, while at the same time it defines medical
error and patient safety more broadly to include overuse, underuse, and
misuse of technology. The report also focuses heavily on medication
errors, but with attention only to medical devices that may play a role in
these types of adverse events—it does not devote attention to the non-
medication related device arena. We propose here that it is necessary to
broaden that perspective. Following is a discussion of these research
agenda questions along with related issues.
Research to Date
There has been little in the way of formal, published investigation of
adverse device events. As noted above, there have been critical incident
studies looking at anesthesia management failures and errors (13, 14, 15,
21, 28, 33, 34, 51, 57), some of which deal with adverse device events.
Beyond this, there have been several studies on adverse events in general.
Retrospective medical record review has been used to identify the
incidence of adverse events, the epidemiology of adverse events, and
potential quality improvement or risk reduction methods and techniques (5,
7, 8, 47, 48, 53, 61). Certain hospital characteristics have been associated
with the increased risk of adverse events or adverse events due to
negligence, e.g., ownership, discharges, minority population, payer mix,
operating costs (8, 11). Prospective observational studies have also been
conducted (2) in this area.
Studies of closed malpractice claims provide data on adverse events in
general, but little on adverse device events. The American Society of
Anesthesiologists (ASA), for example, has analyzed closed malpractice
claims since 1985 (35, 42). Other studies have examined closed claims in
other specialties (41). Several professional liability carriers conduct
proprietary closed claims studies with their own data. The methodology for
closed claims studies has some common elements. For example,
reviewers examine preexisting files related to a malpractice case, which
vary in scope from a few pages to several hundred. They complete data
collection forms that include objective factors (i.e., patient demographics,
type of anesthetic agent) and subjective factors (i.e., was the appropriate
standard of care met, would better monitoring have prevented the event).
The abstracted data are examined for accuracy and then analyzed.
However, the purpose of closed claims studies is to identify areas of
liability for a particular specialty and, in some studies, to determine the
extent to which substandard care contributes to injury (35). Future studies
on medical error with healthcare technology should focus instead on
causes and errors, not allegations of liability.
Return to Contents
To what extent do medical devices and related information systems
contribute to medical errors (including medication errors), especially
within the technology intensive medical specialties?
Despite widespread concern and the implementation of government-
mandated reporting systems, there has been little in the way of formal
published investigation of adverse device events. Although FDA
occasionally issues safety alerts, it does not provide the user community
with any feedback on device evaluations. Nor does it provide reports of
trends or patterns from its database (GAO. Medical Device Reporting.
Improvements Needed in FDA's System for Monitoring Problems With
Approved Devices. January 1997.(50)) An understanding of the causes of
medical device accidents and the errors and failures underlying them, is
the key to prevention—learning a good lesson from a bad event.
The number of medical devices is highest in the areas of medical specialty
mentioned above, in which a single procedure can involve dozens or
hundreds of devices, components, and accessoriesboth disposable and
reusable devices. In these special healthcare areas, there is a complex
interplay among person and machine, the behaviors of individuals and
specialty care teams, and organizational structures and processes.
Research on medical error should establish an evidence base, similar to
that done in anesthesia, for medical technology used in these areas of
clinical specialty, and other areas where one would expect to find
accidents or adverse events involving the use of medical devices,
equipment, and systems. Doing so will further help establish denominator
Most adverse device events occur from multiple causes and/or devices.
For example, there is likely a strong positive correlation between: 1) the
use of electrosurgical units during head and neck surgery, and 2) surgical
fires (17, 23, 26, 27). In such cases, it is unlikely that the electrosurgical
unit or the anesthesia equipment malfunctioned, but it is likely that the
interaction of the electrosurgical spark, flammable surgical materials, and
patient hair in the presence of the oxygen delivered from the anesthesia
machines is a primary cause of many such accidents. [There are other
third-order reasons why the spark resulted in ignition of patient hair or
surgical drapes, such as the inadequate understanding or dissemination of
fire prevention principles.] Knowledge of multiple causes is likely to induce
more lasting change (46).
The results of research into this question will also allow development of
recommendations that would reduce the likelihood of "design"
deficiencies in the broad sensefor both the user community and the
vendor community. Users, for example, may need to improve procurement
or other management processes for specific devices. Manufacturers, for
example, may need to improve human factors or labeling. Medical and
nursing schools will likely need to develop curricula that address the
fundamental physics and mechanics of medical technologya process we
have pursued for three decades and which medical educators still ignore.
Question 1 and the Denominator Problem
Denominator data is generally lacking related to medical errors with
medical devices, with the exception of a few specific areas that have been
well studied over the past two decades, such as anesthesia and
cardiopulmonary bypass perfusion. Medical device accidents are uniquely
complex to investigate. General estimates suggest that up to 90% of all
errors in medicine are caused by human error (6), with human error in
medical device accidents accounting for 50-70% of all device related
accidents (52). However, at this time researchers cannot readily go to the
published medical literature to determine denominator data related to
medical errors involving medical devices. Lacking are data relating to the
incidence of injuries per device type, the number of mishaps per the
number of patient's treated with a particular device type, or the number of
injuries related to the number of uses of a device.
Lacking good denominator data, it is difficult to estimate the percentage of
medical errors that involve healthcare technology. At ECRI, giving
consideration to our databases and our detailed knowledge of the FDA
medical device problem reporting databases (MDR and MAUDE), our
guarded impression is that approximately 5% of all medical errors involve
a medical device, system, or technology as a contributing factor to the
error. Clinicians, regulatory personnel, and other researchers with whom
we have discussed this issue are similarly cautious in ascribing
percentages, but, even lacking data, believe that the percentage is
significant enough to warrant detailed study.
Within the studies of anesthesia mishaps, human error, equipment failure,
and disconnections were specifically studied (14, 15). The percentages of
accidents involving actual equipment failure has been consistently low,
from 4-14% in these studies. However, review of these study's data reveals
that the percentage of user error mishaps involving equipment is much
higher, ranging from 59-62%. These figures exclude the reported accidents
from device disconnections (e.g., breathing circuits and intravenous
medication lines), which, if included, would raise the percentage of medical
errors in anesthesia that involve medical devices even higher.
Perspectives that dismiss research into medical error based on the
observation that patient injuries related to equipment failures in certain
specialties are rare (such as anesthesia) miss the point that valid research
of medical errors related to devices should not focus on device failures,
per se, but on medical errors related to device use and attendant user
errors. A dismissive perspective also, unfortunately, suggests that such
patient injuries and deaths are not worthy of detailed prospective or
Although denominator data is very scant, research into medical errors
involving medical devices is warranted, but on a focused basis, either by
medical specialty or, in some cases, by technology type, recognizing that
the majority of medical device accidents can be attributed to user error
(37, 52). Such focused research in this area will help establish denominator
data for the development appropriate preventive recommendations and for
the assessment of the assessment of their effectiveness.
Analysis of device related mishaps in the demanding environment of the
intensive care setting has yielded some limited data (1, 18, 20) suggesting
that 66% of equipment related medical errors in the ICU were caused by
user error. When considering the clinical relevance of these findings, it is
clear that the perception that "engineers perceive theoretical hazards," or
that the serious device related accidents reported to ECRI or FDA are
"unique, rare, exceptional events," are claims to be rejected. The reality is
quite different, but requires research to better quantify it.
Question 1 and Human Factors Research
The study of system failures and human factors has, over the past decade,
been performed with greater frequency in the medical context (6, 9, 22, 34,
37, 46, 49, 55, 56, 58). Considerable work has already been done in this
arena, much to the benefit of patient safety, and the study of the user-
machine interface continues. However, medical error with medical devices
involves much more than the study of human factors engineering criteria
for the device design. Unfortunately, the focus on healthcare technology
has, as seen in the IOM report, generally been limited to medication errors
with infusion technologies. Future research into medical errors that occur
with the use of medical devices must be extend beyond the bounds of
human factors research, as suggested in the research questions posed
In regard to the medication error issues contained within this question, the
human factors issues related to infusion pump design are very important,
but the extent to which these pumps, or other related devices and
medication order systems, contribute to medication errors is not known.
There is no denominator data because no one has researched the topic.
With denominator data, the healthcare community could perform safety
audits with a medication error reduction focus. From those audits, the
development of recommended policies and procedure related solutions
could proceed, followed by education to hospitals and, ultimately, the
drafting of best practice guidelines. However, this approach hinges on
being able to derive sound data.
The answers to the research questions posed here will facilitate many
patient safety initiatives on the part of users, regulators, vendors, and
other stakeholders. Manufacturers of medical devices would better
anticipate, during the product design process, the errors and injuries
related to medical devices and, thus, incorporate appropriate engineering,
design, and safety controls to minimize the risk of common errors. For
example, a number of studies have shown a high correlation between user
error and mishaps with anesthesia technology (13, 14, 15, 28, 33, 34, 51,
57). Consequently, over the past 15 years, manufacturers of anesthesia
machines and related perioperative physiologic monitors used by
anesthesia personnel have incorporated significant human factors
considerations into the design of devices that protect against user error.
For the user community, such studies have led to the development of
improved administrative guidelines and enhanced organizational
structures within the anesthesia work environment (16).
Return to Contents
What engineering controls, including those based on continuing
human factors research, can be employed to minimize the likelihood
of medical errors?
Innovation in safety features and engineering controls in medical device
technology is driven in part by the feedback that device manufacturers
receive from clinicians, not only during the product development testing
phase but also after FDA marketing approval. Unfortunately, many hazards
that could be mitigated by safety features or engineering controls are not
identified until the post-market surveillance phase, subsequent to FDA
clearance, i.e., when adverse events have occurred and patients have
suffered injury. The answers to this question will serve as additional input
for medical device manufacturers seeking to prospectively identify device-
specific classes of hazards and to "design them out" before their devices
An example of a rapidly developing technology that holds promise in this
area in the near future is the computerized physician order entry system
(CPOES). In the past 5-15 years, error reducing technologies placed into
service over the past 5-15 years have played a large role in the reduction of
errors in a few areas of medical specialty such as anesthesia, pharmacy,
surgery, cardiopulmonary bypass perfusion, and radiation oncology. Some
of the devices and technologies that have been developed and placed into
clinical service in this regard are:
- Anesthesia agent monitors
- Pulse oximeters
- Computerized Physician Order Entry systems (CPOES)
- Drug compounders used for automated mixing of IV solutions and parenteral solutions in the hospital pharmacy
- Electrosurgical unit return electrode monitors (REM)
- Membrane-based cardiopulmonary oxygenators
- Extracorporeal blood flow, blood level, and foam detectors
- Radiologic quality control devices: real-time monitors of delivered dose from linear accelerators.
The development of each of these technologies addressed a specific
patient safety need within that specialty. It is indicated that research into
errors with technology be targeted within the specialties mentioned above
order to be effective. Broad-based research approaches are not likely to be
Recommendations for patient safety developed from the research in the
area anesthesia management have not been the only success. Systems-
based approaches to the management of healthcare technology have been
developed and implemented (28, 29, 30). These technology and risk
management related strategies have helped ensure hospital selection of
equipment appropriate for the use environment, maintained equipment in
proper working order, and rapidly alerted clinicians and administrators to
devices subject to recall or hazard notifications.
Return to Contents
Proceed to Next Section