This information is for reference purposes only. It was current when produced and may now be outdated. Archive material is no longer maintained, and some links may not work. Persons with disabilities having difficulty accessing this information should contact us at: https://info.ahrq.gov. Let us know the nature of the problem, the Web address of what you want, and your contact information.
Please go to www.ahrq.gov for current information.
Transcript of Web-assisted Audioconference
This Web-Assisted Audioconference, broadcast on April 15, 2003, was the second
in a series on bioterrorism conducted throughout 2003 via the World Wide Web
and telephone. It was designed to address the issues and activities associated
with the planning and conduct of disaster drills and on new tools being developed
to assess health care facility and clinician readiness. The User Liaison Program
(ULP) of the Agency for Healthcare Research and Quality (AHRQ) developed and
sponsored the program.
Cindy DiBiasi: Good afternoon. Welcome to Disaster Planning Drills and
Readiness Assessment. This is the second event in a series of Web-Assisted Audioconferences
on bioterrorism and health system preparedness. These events are designed for
State and local health policymakers and health systems decision-makers. This
series is sponsored by the U.S. Department of Health and Human Services Agency
for Healthcare Research and Quality, often referred to by the acronym "AHRQ"
or AHRQ. My name is Cindy DiBiasi and I will be your moderator for today's session.
The context for these calls is clear and compelling. Bioterrorism represents
a significant public health threat to the United States. Addressing this threat
requires the rapid development of Federal, State and local capacity to respond
to potential bioterrorism events. This means improving the abilities of both
our public health and healthcare delivery systems to detect and respond to
such threats. It also means ensuring that public health, health systems, facilities
and providers communicate and coordinate effectively with one another as well
as with other related systems such as emergency preparedness and law enforcement.
The intended audience for these calls are State and local health policymakers,
program administrators and health system decision-makers, all play an essential
role in these efforts. Within their own jurisdictions, regions or delivery
systems they must develop capacity and coordinate efforts across public health,
healthcare, law enforcement and related systems. It is therefore extremely
important that they have information about emerging health services research,
promising approaches and available tools that can assist them in the development
of readiness plans.
In addition to today's event, three more calls will be conducted as
part of this series. The next call, scheduled for June 17, will focus on surge
capacity assessment and regionalization issues. The fourth call on October
21 will examine information and communication technology as well as surveillance
and monitoring systems. The topic for the fifth call will be selected at a
later date. I will tell you more about these calls later in the broadcast,
but right now let's turn to today's call.
We will be discussing the development of new tools to assess how prepared
healthcare facilities and clinicians are to respond to a bioterrorism event.
We will also look at issues and activities associated with the planning and
conduct of disaster drills.
Let me begin by introducing today's panelists. In the studio with me
I have Sue Skidmore, a nurse and an associate at Booz Allen Hamilton. Dr. Howard
Levitan, an emergency physician and consultant from Disaster Planning International.
Dr. Gary Green, an associate professor of emergency medicine and pathology
at the Johns Hopkins University School of Medicine. And Mary Massey, disaster
coordinator at Anaheim Memorial Medical Center in Anaheim, California. Welcome
Before we begin our discussion I would like to tell the audience a bit about
the format of this audio conference. First we will talk with our four panelists,
then open the lines to take your questions. We will give instructions on how
to send these questions to us later on in the program.
In the meantime, if you experience any Web-related technical difficulty during
this event, please click the "help" button in your window to troubleshoot
your Web connection. If it appears that the slides are not advancing, you may
need to restart your browser and log on again. If you are on the phone, dial "*0" to
be connected to technical assistance. Also if you have any difficulty with
the audio stream or if there is an uncomfortable lag time between the streamed
audio and slide presentation, we encourage you to access the audio via your
phone. The number is 1-888-496-6261. Give the password "bioterrorism" to
be connected to the call.
Now I think we are ready to discuss today's topic, Improving Health
Systems Preparedness Through Readiness Assessment and Disaster Planning Drills.
Let's begin with Sue Skidmore from Booz Allen Hamilton and Dr. Howard
Levitan from Disaster Planning International.
During the first phase of their AHRQ-sponsored project, Sue and Howard developed
a questionnaire to help hospitals and health systems assess their level of
preparedness and their capacity to respond to bioterrorist attacks. Howard,
can you tell us more about this initial questionnaire and why was it developed
and what the most important findings from the assessment were?
Dr. Howard Levitan: Sure. The initial questionnaire was developed for several
reasons. The first was to begin assessing hospitals' readiness for bioterrorism
based on newly developed benchmarks for preparedness. Basically we wanted to
see where hospitals were in their planning process based on some preconceived
norms. Second was to gather information so we could begin gauging the cost
of preparedness and finally it was to develop and collect data that could be
used for computer modeling purposes.
During the project's first stage, we assessed four hospitals to determine
their readiness for a bioterrorist incident. Two of the hospitals were in Georgia
and two were in Maryland. Several findings really stood out. In general, it
seemed that the hospitals were working to improve their readiness, but each
one seemed to be focusing their efforts on different areas that they thought
was applicable to bioterrorism. I think this occurred because there was really
no nationally accepted benchmarks of adequate preparedness on which to gauge
or direct their readiness efforts. In addition, there was no mechanism in place
to share best practices or useful solutions between hospitals. Most of the
bioterrorism plans that we reviewed to develop our assessment tool placed the
majority of their emphasis on the management of specific biological agents
rather than really addressing a number of regional and planning issues. Such
issues as hospital staffing; how to increase inpatient and outpatient treatment
capacity; the stockpiling of resources; internal and external communications;
security, which is always essential in these events; the ability to provide
mass prophylaxis and vaccinations; tracking of local and regional bed utilizations;
the problem of emergency department overcrowding and the interaction with the
Finally, we found out that really across the board administrative support
was key to bioterrorism readiness as well as preparedness in general.
Cindy DiBiasi: Sue, how does this new questionnaire project differ from the
Sue Skidmore: The addition will be developing the objectives measures that
users need in order to interpret the results of the questionnaire and later
in the project will develop some planning templates that will be useful to
facilities and systems and also identify best practices along that preparedness
Cindy DiBiasi: It seems that there are a lot of different assessment tools
and preparedness questionnaires available throughout the country. Why do you
think there is so much variation?
Dr. Howard Levitan: That is a good question. You are correct, there are a
number of assessment tools available. We actually reviewed over 12 prior to
the development of our first assessment tool for AHRQ. I think it has occurred
for several reasons.
First, all 50 States as well as several locales were tasked to assess their
readiness for funding purposes, and as a result either developed their own
assessment tool or simply modified an existing one to reflect their unique
local and regional issues.
Secondly, a number of State and Federal grants were also awarded to develop
an assessment tool. As a result, a number of assessment tools were developed
to gather data on readiness. Yet there are still no established objective measures
of bioterrorism preparedness for hospitals nor was there any generally accepted
attack scenarios that hospitals could use as a basis for their preparedness.
They have prepared for 20 patients, they prepared for 100 patients. We just
In addition, most of the assessment tools have not been evaluated for validity
or in other words evaluated to determine if the questions that they ask or
the information sought are true predictors of preparedness. In my opinion this
aspect alone is the biggest challenge and one of the most important components
of this project.
Finally, most of the assessment tools failed to obtain any buy-in from various
government agencies, national organizations or regulatory bodies.
Cindy DiBiasi: Well since many of the tools out there probably have not been
evaluated to see if they are collecting the information expected or if buy-in
has been obtained from key groups, how are you approaching the development
of your assessment tools to ensure that you are collecting the right information
and getting the buy-in you need?
Sue Skidmore: What we did in this project is we took our original questionnaire
and we then took questionnaires from across the country that were either very
well known, very comprehensive, well done, or had some other reason why they
were used in a specific area. We took all of those and pitted them against
each other and assessed the questions, categorized the questions and rated
them in terms of whether it was pertinent to hospital disaster planning in
general or whether it was relevant specifically to bioterrorism. We weeded
everything out and focused just on bioterrorism. Then we brought in a working
group that was composed of those various government agencies and national organizations
and regulatory bodies and we asked them to walk through all of the categories
and questions with us and to really help us focus on that minimal level of
Finally, we engaged a market and national survey research firm who has experience
and is focused on just developing healthcare surveys and questionnaires in
order to help us really focus the questionnaire, the question design on content,
clarity, design and administration in order to make sure the questions we asked
were tight and got the information we asked.
Cindy DiBiasi: Let's talk a bit about that. How are you specifically
going to redesign your questionnaire to make it easy to use but yet still specific
enough so that it addresses the issues that are most relevant to hospital bioterrorism
Sue Skidmore: The biggest challenge in the questionnaire design is to craft
questions that allow users to get the information they need without making
it too long or too cumbersome.
In addition, the HRSA grantees need a questionnaire that closely follows the
priority areas that they are being asked to benchmark themselves in. So we
are keeping our categories consistent with the HRSA priority areas and in addition,
we looked at the question design itself and we chose only questions that only
focused on issues that are under the responsibility and control of the hospital
leadership. Questions that pertained to the unique response requirement of
a bioterrorism event and that did not duplicate the generic mass-casualty protocols
of an existing facilities emergency management plan and we limited regional
issues to those activities involving the hospital participation and role. Questions
were redesigned for benchmarking purposes in that the answers would not be
open-ended answers; there would be less data collection, they would be more
scaled so we have "no we don't do this at all", "yes
we do this a little" and "oh boy, we have a best practice over
here" so this is scaled along the continuum.
Cindy DiBiasi: Howard, in your opinion, what makes one hospital better prepared
for bioterrorism compared to another?
Dr. Howard Levitan: Well, that is the big unknown and the challenge that AHRQ
has tasked us with. However, instead of really focusing on comparing one hospital
to another, we sought to determine the essential planning issues that ever
hospital must address.
During phase one we began the process by developing a list of preparedness
criteria based on routine joint commission requirements, lessons learned from
natural and man-made disasters that we gleaned from the literature and from
the opinions of our working group who represented really most of the major
healthcare organizations in the United States. Our objective now is to reach
consensus on identifying the critical components of bioterrorism preparedness,
then on establishing indicators of preparedness along a continuum rather than
really delineating good, better or best preparedness.
Cindy DiBiasi: What areas does your questionnaire focus on, Sue?
Sue Skidmore: After reviewing all of the tools and questionnaires that we
compared our questions against, a dozen or more, we came up with twelve main
categories, regional linkages, bioterrorism planning and structure, training
and exercise, triage diagnosis and treatment and so on. We found that they
are actually closely aligned or will ultimately align with the HRSA priority
The current timeline for the questionnaire is that the revised version will
be ready to pilot near the beginning of June and the plan is that a small sampling
of HRSA grantees will be asked to volunteer to participate in the pilot questionnaire.
Data and feedback will be obtained and used to make further revisions and then
the final questionnaire will be delivered to AHRQ.
Cindy DiBiasi: OK. Thank you Sue and Howard. We will get back to you with
But first I would like to turn now to Dr. Gary Green from the Johns Hopkins
University Evidence-based Practice Center. Gary, I understand that you are
part of a group that has been working on two AHRQ-sponsored projects and they
are evaluating the existing evidence about healthcare worker and institutional
preparedness for bioterrorist events.
Tell me, do you think there is adequate evidence out there concerning effective
planning and preparedness efforts to guide those responsible for hospital disaster
planning or does every hospital and health system have to come up with their
own method for disaster drill planning and evaluation?
Dr. Gary Green: There is some evidence out there to guide us, but it is extrapolated
information based on the evidence in the general disaster response literature.
Our Evidence-based Practice Center's first evidence-based report on this
topic examined the existing literature on training clinicians for a public
health event relevant to bioterrorism preparedness. The evidence report was
released last year and is now available on the AHRQ Website.
A second project, which is not yet complete, examines, evaluates and summarizes
the current evidence specifically concerning hospital (unclear). Without going
into detail, I can tell you that this is very much a field in infancy and there
really are not yet any high quality scientifically-based, peer reviewed publications
addressing either disaster planning or disaster drills specifically related
to bioterrorist events.
The good news is we did find that there is a rapidly growing body of evidence
concerning disaster preparedness in general and that much of this literature
is also applicable to bioterrorism preparedness.
Cindy DiBiasi: Gary, specifically what does the literature tell us?
Dr. Gary Green: Well, general disaster preparedness literature is also in
a relatively early stage. However, there is some limited conclusions that can
be drawn from the existing literature that the Evidence-based Practice Center
First, the basic components of an adequate disaster response system have been
defined. The basic steps needed for disaster preparedness capacity building
have been established. In addition, a variety of training and assessment techniques
have been described and have already been successfully used. However, the terminology
in the field has not yet been standardized nor have the best techniques and
practices been established. For example, even the term "disaster drill" means
different things to different people. Some use the term to describe all disaster
simulation including tabletop and computer simulation of disaster events, while
others use the term in a more specific way only referring to activities if
the event is physically operationalized and mock patients are used.
Cindy DiBiasi: What steps should a hospital or health system take to improve
their disaster response preparedness?
Dr. Gary Green: I will start by saying that there is no cookbook recipe that
every institution can follow that will lead to the product of adequate preparedness.
However, based on existing literature and based on my own experience, I can
outline a basic framework that will help each institution determine what they
need to do given their individual environments and resources.
One way to look at this process for disaster preparedness capacity building
is to consider how similar it is to a continuous quality improvement initiative.
One of the principles of CQI is that the project is never completed. In other
words, no institution ever reaches a total State of preparedness for any disaster.
Rather disaster preparedness efforts should be viewed as a continuous cycle.
The basic steps within the cycle are as follows. First, an interdisciplinary
team has to be assembled. That includes all the key stakeholders. That team
will then review the current resources as well as the strengths and weaknesses
of their individual health system with regards to disasters. That will include
both looking at the history of previous disasters that that institution has
responded to and doing an objective assessment of that response as well as
trying to do an objective assessment of the risk of future events based on
the available evidence.
The next step would be to develop a detailed written disaster response plan.
To this end, to disseminate and practice that plan, disseminate the plan throughout
the institution and practice it with all the staff through both educational
interventions and through disaster drills.
As your education and training progresses, you then have to evaluate the knowledge
and skills of the staff using objective data gathering techniques and that
will lead you to a determination of the specific strengths and weaknesses of
the staff, of the disaster plan and the training techniques. Based on that,
your next step is to reengineer the disaster plan based on that data and then
go back and modify your educational interventions and training to better target
those areas of weaknesses.
As seen in this slide that is then a continuously repeating cycle with training,
evaluation and reengineering. This cycle can be applied at the institutional
level or the departmental level or the regional level. Of course the length
of the cycle can vary according to the needs of the individual institution.
Cindy DiBiasi: What are the differences between preparing for conventional
disasters and preparing for bioterrorism events?
Dr. Gary Green: As illustrated in the next diagram, preparedness for conventional
disasters is really the foundation for bioterrorism preparedness. You cannot
be prepared for a bioterrorist event or another non-conventional attack or
event without first being prepared for a conventional disaster. This is because
most of the knowledge, the skills and the resources needed are the same. Also,
healthcare institutions are likely to be called on to respond to conventional
disasters far more often than non-conventional disasters. Therefore, putting
all the pieces in place for general disaster preparedness is the first and
most important step for preparedness for responding to a bioterrorist event.
So, in this schematic, we can see that the solid circles represent the response
to a conventional attack, either at the hospital level or in pre-hospital level,
whereas the dotted lines surrounding them represents the additional activities
and preparedness efforts necessary to respond to an attack or an event based
on a biological agent or radiation or chemical event. Also included is the
relationship between the pre-hospital team and the hospital response and how
an Incident Command System really is important and effective throughout all
of these different components.
Cindy DiBiasi: You said earlier that the basic building blocks of a disaster
response system have been defined. Can you elaborate a little bit on that?
Dr. Gary Green: Yes Cindy. The fact is that to many of us disaster response
is by its very nature complex and confusing. One of the reasons that the task
of preparing an institution for such an event seems to be so overwhelming is
that it is very difficult to break down into manageable components. The many
simultaneous activities necessary to respond to a disaster all have to happen
simultaneously and have to be intricately coordinated. To deal with this, there
have been several attempts made by a variety of organizations and groups to
come up with a basic structure for a disaster preparedness effort and to break
down disaster response activity into more clearly defined and understandable
As I mentioned earlier, there is still no agreement on the exact terminology
and different groups stress somewhat different boundaries around the various
response components. But the basic building blocks are more or less agreed
upon. In a disaster, each one of those components must be actively managed
by a designated commander according to a pre-determined action plan.
Perhaps the most important component is the Incident Command System. This
is also referred to as an Incident Management System or some people still use
the term "Command and Control." To a large extent, this component
defines the boundaries and the functions of the other response systems.
Another important component is system integration, which is absolutely key
and largely refers to communication functions. Both communication functions
within institutions as well as communication between the institution and outside
Logistics refers to supply and management of materials, facilities as well
as transportation of patients, providers and materials. Security, as mentioned
earlier, is critical to all the other components in order to keep the other
components working the way they are planned to. Clinical care, human resources,
public relations of course each have very specific requirements that have to
be addressed as well. Then individual institutions may also categorize other
components according to their needs.
Cindy DiBiasi: Gary, what do we know about effective techniques for training
healthcare workers to be prepared for disaster?
Dr. Gary Green: Well, based on our EPC's first evidence report on this
topic we know that a variety of techniques have been used to train both clinicians
and other members of the healthcare system in disaster preparedness. Unfortunately,
very few of any of the reports about this technique applied scientifically
rigorous evaluation methods so there is really only very limited data concerning
which of these techniques has been successful.
There is also almost no objective data about the relative cost of the various
techniques. The training techniques that have been prepared, that have been
reported in the literature, include the traditional educational techniques
that we are all familiar with, lectures, discussions and use of audio-visual
aids and written materials.
Standardized or smart patients. Standardized patients or use of smart patients
refers to training healthcare workers to be mock victims. That way during an
interaction with providers, mock victims can evaluate and teach the providers
as they are treated. The use of smart patients has been found to be very acceptable
to physicians and it is an effective technique for one-on-one training. However,
it is questionable whether it is useful for training large numbers of people.
It may be very cost ineffective; it is unclear whether the cost would be prohibitive
to train a large number.
Teleconferencing, the satellite broadcasting, has also been used. Reports
of using this as an educational technique for disasters has been recorded in
the literature. This seems to be very useful for reaching large numbers of
trainees and seems to be as effective as traditional classroom learning based
on what evidence there is out there.
Tabletop exercises is another technique that is in use. It seems to be used
quite commonly. A tabletop exercise is a sort of theoretical drill without
physically operationalizing patients or equipment or personnel movement. It
is usually sort of focusing on the Incident Command System and system integration
components. It has been successfully applied to teaching clinicians bioterrorism
preparedness specifically. It seems to be a very useful technique although
this technique might be best used when combined with other techniques as part
of a comprehensive plan.
Computer simulation is something that there is a lot of buzz about, but there
has really only been one or two evaluations of computer simulations have been
recorded in the literature. There is very limited data out there. However,
the authors of those papers have suggested that it may be able to replace expensive
drills and also might allow identification of deficiencies in disaster planning
Then of course there is what we all usually all refer to as disaster drills.
Disaster drills are the most generally accepted training tool and they are
really sort of at the cornerstone of disaster preparedness efforts. There is
very significant collective experience in their use. Unfortunately as mentioned
before, there is very little standardization and very limited data concerning
objective evaluation of disaster drills. Drills have been shown to improve
knowledge of the disaster plan and they have also been successfully used to
identify problems and plan execution.
Cindy DiBiasi: Gary, we are going to get to come back to you for questions
as well. But before we do that, we have been talking a lot about disaster planning
drills and what the literature tells us about them. Let's first go to
Mary Massey, the disaster coordinator at Anaheim Memorial Medical Center, who
has been at the center of many drills. Mary, if you could share some of the
insights that you have gained from being at the ground level. My understanding
is that you and your colleagues at Anaheim Memorial Medical Center have been
involved in numerous disaster drills.
Mary Massey: We have been holding disaster drills for over 20 years. Home
of some high-venue sights like Disneyland, Arrowhead Pond, Edison Field, Anaheim
Convention Center, as well as our share of earthquakes and fires, we found
that it is important as a community that we have been practicing these drills
for the different scenarios.
We have been practicing it at different levels too. Not only department-specific
type drills, but hospital-wide ones. We move on to community-wide ones. We
are an MMRS city, Metropolitan Medical Response System city. There are three
of them within Orange County, Anaheim, Santa Ana and Huntington Beach, where
the government came in and helped us prepare for bioterrorism. At a minimum
we have to be prepared for 2,000 chemical victims and 10,000 biological ones.
Going larger than that, we practice in a net, our county is divided into separate
nets, that practice a large, community-wide drill once a year. Our net has
17 hospitals and several provider agencies, police and fire.
Then a step above that is the annual Statewide drills. This past year we did
a radiation drill, which was a new one for us. We found the change shifting
the last few years from more manmade-type disasters up into terrorism-type
focuses. So we have found that we have had to learn how to deal with mass-casualty
decon, how to handle fear and large levels of the worried well.
Cindy DiBiasi: Based on that 20 years of experience, what would you say the
key lessons are in getting a community started in planning a drill?
Mary Massey: The first thing, it sounds kind of simple, but you need to figure
out what you are trying to do. What is the objective you are trying to answer
in this drill and then figure out a way to measure it. So the way we do it
is we make a universal critique for everyone that measures what our objectives
are we stated. Then we bring all the players together. A community-wide drill
requires law enforcement, fire, EMS, public health, DWP, hospitals, other healthcare
organizations and the vital industry. Sometimes I think we forget who is in
our community. We have a high tourism industry in Anaheim so we need to involve
them in it too. This helps you build the relationships that will lead to trust
and cooperation in collaboration.
Cindy DiBiasi: That seems like a lot of relationships to manage. Is everybody
Mary Massey: We have been doing this for so long in Orange County that we
have built really good relationships between the different agencies. Not that
we are always perfect; there is always progress to be made, but we realize
that this isn't always the case. Don't let long-standing differences
or personalities block working together. It isn't a battle to see which
agency is the best; it is a battle to prepare your community to handle your
own citizens. It might by your own agency personnel or your family. The better
prepared you and your competitors are, the better chance of survival.
Cindy DiBiasi: So once you bring everybody together and you build the relationships,
Mary Massey: Then we need to develop a community-wide plan. We have all worked
on having our own agency's emergency plans, but we need to have a plan
that works well when we are working with the other agencies. (Unclear) requires
disaster plans and testing twice a year for hospitals. What we need to do now
is develop an integrated response.
Some of the major points are definitions of roles and responsibilities. We
use HEICS or "HIKES", depending on where you are in the country
for pronunciation, equipment and supplies and communication. This is a good
time to test your redundant communications because you don't know what
you will be able to use when an actual disaster strikes. Some examples are
phones, cell phones, walkie-talkies, and don't forget the low-tech paper,
pencil and runners.
Proceed to Next Section
Return to Audioconference Session