This information is for reference purposes only. It was current when produced and may now be outdated. Archive material is no longer maintained, and some links may not work. Persons with disabilities having difficulty accessing this information should contact us at: https://info.ahrq.gov. Let us know the nature of the problem, the Web address of what you want, and your contact information.
Please go to www.ahrq.gov for current information.
On May 22, 2008, Rosanna Coffey presented the AHRQ Asthma Return-on-Investment (ROI) Calculator on a Web conference. This is the transcript of the event's presentation.
May 22, 2008
1:00-2:15 p.m. ET
Moderator-Margie Shofer: I'm Margie Shofer in the Office
of Communications and Knowledge Transfer at the Agency for Healthcare Research
and Quality. Thank you for joining us for this Webinar on the Asthma Return
on Investment (ROI) Calculator. This Webinar is the second of three followup
events featuring AHRQ tools that were shared at two workshops held this past
December and January. We focused on the State Snapshots at the end of April
and will hold another event June 16 on the preventable hospitalizations county-level
mapping tool. We see all these events as the first step in what we hope will
be a series of follow-on technical assistance opportunities. If, after learning
more about the calculator today, you're interested in further assistance from
AHRQ in using this tool, please let us know.
We're hosting this audioconference in response to interest in the ROI Calculator
expressed by participants at both workshops. We know that some of you attended
one of the workshops, whereas others may have had less time to interact with
the tools. As such, we will spend some time reviewing the basics of the tool.
We will then move on to a demonstration of the tools, will address questions
about underlying data raised at the workshop, and will discuss interpreting
the results for a variety of policy applications. We will conclude with a few
caveats and further discussion.
We would appreciate your active participation, because one purpose of today's
Webinar is to probe further into underlying data sources and other technical
issues you may encounter when using the tool. We also want to explore practical
applications for the tool, collect your suggestions for future tool enhancements
or modifications, and learn how you envision using the tool and for what purposes.
And again, as this last point, we really hope you tell us about the technical
systems that you might need in order to be able to make full or better use
of the ROI Calculator. It's really the purpose of this call today.
We're going to start the Webinar with a brief introduction from Jeff Brady,
Director of the U.S. National Healthcare Quality Report here at AHRQ. He is
an M.D. with a background in primary care and preventive medicine and public
health. And he oversees the National Health Care Quality and Disparities Reports,
the State Snapshots, and all derivative products, which include the Asthma
ROI Calculator. Following his introduction, Jeff will introduce Dr. Rosanna
Coffey from Thomson Reuters Healthcare, who will give today's presentation.
Now I'm going to turn this over to Jeff.
Jeff Brady: Good afternoon everybody. It's
my pleasure to be here today and involved with this WebEx teleconference. As
you'll hear today, the Asthma Return on Investment Calculator can help you
estimate the financial benefits of asthma quality improvement programs in your
State. A real success of the calculator is that it combines information from
the literature with real world data about patients to estimate the net impact
of an asthma care quality improvement program. As you'll further see, it has
lots of information about medical utilization and asthma prevalence that you
can really input to apply this to real world examples.
And so what I'd like to do is introduce not only Rosanna Coffey, but also
Ginger Carls, who unfortunately can't be involved in today's conference but
has been very involved in both the production and the dissemination of the
tool. As you participate further and perhaps engage in some technical assistance
with the tool, if you have any questions that go beyond today's conference,
it's very likely that Ginger will be involved in those.
Ginger's an economist in the Health and Productivity Research Department at
Thomson Reuters Healthcare. She's worked on various projects to evaluate the
performance of disease management programs, as well as health burden studies.
She's currently a doctoral candidate pursuing a degree in the Department of
Economics and Health Management and Policy. For today's presentation, Rosanna
Coffey will lead that. Rosanna is a Vice President at Thomson Reuters Healthcare,
formerly Thomson Healthcare, which before that was Thomson Medstat. Rosanna
and her team support the development of the Asthma ROI Calculator. Rosanna
is a Ph.D. economist with more than 30 years of health services research experience.
So, I'd like to turn it over to Rosanna to start.
Rosanna Coffey: Thank you, Jeff. Good afternoon,
everyone. I want to acknowledge Ginger's hard work on this. She is the energy
and the brains behind this calculator. And we also had other colleagues at
Thomson that we worked with: Ron Ozminkowski, and Greg Lenhart, who was our
Excel wizard, and you'll get to see some of his work later. I'm stepping in
for Ginger since she had a death in the family. If I get stuck on anything,
we will get back to you in e-mail; that is if you ask me questions that I can't
answer, but I hope to be able to answer them all.
So, before we start, I would like to ask you all a question. And that is,
we're going to open the polling option, and we want to know, have you used
the calculator, or are you planning to use the calculator? You should see that
question, and if you just click on "yes" or "no," we'll get the results as
they come in.
So I'm going to continue with the presentation at this point.
The plan for today is to give you a very brief review of the calculator, because
many of you have already seen it and perhaps have used it, and I'm going to
focus on how to use the calculator. The data that you can enter, the data that
underlies the calculator, some of the default information that's already there,
and we'll spend some time later on interpretation of results and looking at
limitations and solutions for those limitations.
You may ask questions at any time through this, by going to the Q&A option
on your right-hand menu. Jeff Brady is going to watch those questions for me,
because I'm going to be busy concentrating on what I'm trying to say to you.
He may interrupt me, if there's a clarifying question he thinks we ought to
address as I'm going through.
Let's start with a review of the calculator.
The purpose of the calculator is really to help policymakers-primarily State
policymakers-although private health care can use this tool too. The purpose
is to help with program design for some kind of asthma care quality improvement.
The calculator helps to estimate the financial gains or losses that you might
observe if you institute a quality improvement program. And keep in mind, financial
gains may not be the only reason that you may want to institute a quality improvement
program, but this tool, at least, will help you get a handle on the financial
The calculator has summarized a huge literature pool; 52 studies are used
behind the calculator, to come up with impact estimates. When you look at that
literature, it is no mean feat to summarize it. People do analyses differently;
they look at different aspects of health care costs. In most of these studies,
the outcome that they were looking at was the change in utilization, and there
were very few studies that attempted to convert that into what that means in
terms of cost. So we did that translation for you.
I see we have the results of our poll. How cool. We're doing this mainly because
I wanted to do it, to see how it would work. And we see that we have 52 percent
of you who have used the calculator, which is great. So we hope that you will
ask your questions. If you have them, we have two places in the presentation
where we ask for your questions, so we're going to address them at that point.
Back to what I was saying: The calculator's purpose, also, is to help you
focus on particular populations that you're interested in.
The message behind the calculator would boil down to something very simple.
It's not quite this simple, but just to get it understood, we take the evidence
that comes out of the literature, which is primarily utilization information.
We add to that external information on cost of health care. And the combination
of the two gives us the financial impact, tied into a lot more information,
which I'm going to show you as we go through this.
Let's define what we're talking about. We're talking about asthma care programs
that typically follow the national asthma education and prevention program
guidelines. And those guidelines focus on patient education, as well as provider
education. On the patient education aspect, we're talking about self-management
of the disease, helping people with asthma understand how to avoid the triggers
that cause asthma attacks, how to anticipate problems, and how to use their
medications appropriately. On the provider side, we're talking about providers,
and getting them to make accurate diagnoses, to prescribe medications appropriately,
and also to monitor their patients' success and their attempts to self-manage
their disease, as well as to provide patient education.
So in what we're talking about, we have combined the literature. Some programs
may focus on different aspects, one on patient education, another on provider
activities and education. But we have lumped them all together. So in some
sense, we have an amalgam here of all of these different types of activities
that have gone on.
So, we're going to turn now, to how to use the calculator, and we're going
to focus on the data.
The data that you need to use this calculator are four main types of information.
You need to figure out who the eligible asthma patients are. You need to know
something about their baseline use of health care and the costs associated
with that. You need to have some evidence around what happens if you implement
an asthma improvement program-that comes directly out of the literature. And
then you want to know what it's going to cost to implement a program, and we'll
talk about that.
So, start with the eligible asthma patients. You have two options. One, you
can calculate the number of eligible patients using your own data. If you are
a Medicaid agency, or you're responsible for a State employee benefit program,
or maybe a high-risk pool, you may have some information on people in that
population. Maybe information from medical claims. You can, and you should,
use your own data to populate the calculator.
If you don't have that information, you can use the default information that
we provided. So let's assume for a minute that you have the information and
you're going to calculate the eligible people with asthma from your own data.
One of the very important things you need to do is think about which asthma
patients you're targeting, and you need to analyze your data, or tabulate your
data for the right population that you're interested in.
You may be interested in any patient with an asthma diagnosis. And that's
the first bullet here, under "defined criteria," or you may want to target
patients who have persistent asthma. We did include both of these, as well
as the third option. But the patients with persistent asthma, we defined based
on the NCQA HEDIS definition, and here's the Web site for finding that. The
other place that you'll find it is in our detailed report.
If you're using the calculator, you should have a copy of our detailed report.
It tells you everything that we had to do to come up with these estimates and
lists all of the literature with abstracts on all of the studies that went
into the calculator. And in tables 8 and 10 of that detailed report, we have
the definition for persistent asthma. The third category that we give you is
an option to choose if you want to target those very high-cost patients. You're
likely to find patients with persistent asthma and an acute health care visit
to an emergency department or a hospital within the year.
To use the calculator to estimate the eligible asthma patients, you would
need the percentage of enrollees in each age and gender cell, or you'll see
that in the calculator. And the percentage of enrollees by each race and ethnicity
is available if you're looking at Medicaid. And you'll have the total number
I am going to jump us to a demo.
Thank you, Jose, for reminding me that I did have a question. But I wanted
to ask our participants again, have you tried to enter data into the calculator?
If you would go to your polling on the right-hand side and answer that question
for us, we'll have a sense of how many of you have actually tried to do this.
Now, I'm going to share my desktop with you.
In a few seconds, you should be able to see the Asthma Care Return on Investment
Calculator. You can see it is in Microsoft Excel, and at the top are a number
of tabs and these really are the main aspects of the calculator: the population
sheet, a participant sheet, the baseline cost and utilization data, program
impact information that comes out of the literature, the program costs where
you may want to enter actual values for the cost of the program you're considering,
and the results sheet.
We have a lot of information in the appendix. Like I said before, we also
have more detail in a paper.
Let's look at that population sheet for a second. You see you have an option
here. You can look at Medicaid as a population; you can go to the employer-sponsored
insurance population, or even State employees and their dependents, if you
choose. And you also have the geographic option-you can select your particular
State. If you're from Alabama, I'm selecting you now. And you can see the size
of the population went down, and distribution of age, gender, and race changed,
and this is for Medicaid in Alabama. And it was about 2005 data. Things may
have changed; you may not think that this reflects you and Alabama very well.
So you would want to enter the size of your population, and you may want to
even enter your age, gender, and race distributions if they have changed.
I am going to go back and select the Nation here to show you the participant
page. This is where you make some choices. Are you targeting children? Are
you targeting adults? If it's both you keep them clicked. If you really just
want to look at children, take off the adult. What asthma severity are you
looking at? Is it all asthma patients, persistent asthma only, or persistent
asthma with acute visits? When I choose the last one, you see the eligible
patients go down. You also have the choice of making a judgment about how many
people that you target are going to participate in this program. We have put
in 25 percent, being very conservative. You may think you'll have a higher
number of people who will be willing to go through the program, and you want
to change that.
Let's jump back to the slide presentation.
As you see here, we have the option of selecting asthma severity, the type
of coverage that you're dealing with, or the population. The prevalence rates
that are in the calculator come out of a Thomson Reuters data product, which
is the MarketScan® Medicaid or MarketScan® Commercial Database. And
those prevalence rates are given as I showed you by the age and gender, and
also race/ethnicity if it comes out of Medicaid. There is no race/ethnicity
in most commercial claims, and we don't have it in ours. Also, we have made
all those calculations for different asthma severity levels.
Now the next piece of information you will see, looking at the calculator,
is the baseline utilization and cost information. This is the big cost that
your population is incurring before you institute an improvement program. And
the data fields relate to all the components of cost that you ought to consider
when evaluating an asthma quality improvement program or you're thinking about
designing one. What costs might be affected by your program?
So we have the per-patient per-year estimate of emergency department visits
or hospital stays or outpatient visits. On the cost side, we're looking at
the cost per visit, or the cost per stay for each of these components, as well
as the asthma medication costs per patient per year, and the asthma ancillary
costs, such as lab, imaging, and pulmonary function tests. Immunizations are
even in here.
Notice that we have paid amounts. We're not talking about charges. We are
talking about what was actually paid for medical claims. You have an option
as to whether you include the patient's copayment. The other data field you
have as part of the baseline is number of missed work or school days. This
is an option for you if you want to estimate the productivity gains from instituting
Our advice in calculating estimates for these data fields if you're putting
in your own data: You need to decide which use and cost components you want
to include. We recommend you be comprehensive. If you've got information on
one department, like emergency department, use what's in the calculator for
the other components. Just make sure that for the emergency department you
define and calculate your use and cost piece in the same way that you would
set up the population. That is, if you're coming up with utilization rates
on emergency department visits, and it's for the people with persistent asthma,
make sure you've made that selection at the front-that you're looking at all
the costs in our calculator that relate to people with persistent asthma. That's
one of the most important things about consistency in using this tool.
So we're going to jump now to share my desktop with you again, and we're going
to look at the baseline data. Here, you do have an option to select asthma
treatment costs, or any treatment costs for anyone with asthma. All of these
light boxes are data that we have calculated from the MarketScan claims data.
I believe we're still selected on the Medicaid population. So these all come
out of our MarketScan Medicaid claims data: the emergency department visits
per patient, the hospital stays per patient, and on the right-hand side, the
cost of an emergency department visit and hospital stay. If you have information
on the baseline for your population, you really should enter that in here to
get some idea about how a quality improvement care program will work for your
Also note here on the top right side. You can take two different perspectives.
You can take just the perspective of your program or plan-for example, how
much Medicaid will save. Or you can take a social perspective and include the
patient's cost in the calculation.
Let's jump back to our slides.
Let me just run through quickly what is behind that baseline information.
I told you about MarketScan claims. There are eight Medicaid States; they are
geographically dispersed. They represent about 7 million covered lives. On
the employer-sponsored health insurance, the private health insurance component,
if you would select that, back at the beginning of the calculator, the baseline
information would relate to over 100 large, self-insured employers, about 15
million covered lives. These employers are located all over the United States.
But one thing to keep in mind, when you're talking about insurance coverage
by large employers, they are generally more generous plans than smaller employers,
or certainly than individual coverage.
For the missed schooldays, or workdays, we borrow an estimate. It's a total
U.S. estimate that comes out of the National Health Interview Survey for 2003.
And we evaluate those missed work and school days, using the average wage rate,
that comes out of the Bureau of Labor Statistics. That particular wage rate,
we have available by State. So we apply the State's average wage rate when
we're coming up with these cost estimates. For people who are on Medicaid,
we use the Federal Poverty Level for the value of a missed workday.
Now we're finished with the baseline.
Let's talk a bit about those impact estimates that come out of the evidence.
The data fields that we have in the calculator relate to the number of asthma-related
hospital stays, ER visits, outpatient visits, payments for prescription drugs
in the retail setting, for ancillary services, as well as those numbers of
missed work and school days. Where do these data come from? This is the big
effort that we undertook. The results come from our meta-analysis of the literature
and that is what is in the calculator.
You may have estimates from a pilot test that you've already done on some
asthma improvement program, and you may want to enter your own pilot test information
in here. Or you may know about a study that's more recent than the ones we
looked at. Our review ended in March 2007. If you know about a new study and
you want to see how that plays into the calculator, you can enter those data
in as well.
Jeff Brady: This is Jeff. Can I interrupt
at this point for a moment?
Rosanna Coffey: Yes.
Jeff Brady: One question that I think fits
into this part of your presentation.
Rosanna Coffey: Absolutely.
Jeff Brady: One question was about the paid
amount for facilities for inpatient care and the fact that that varies, and
what the default amount is. Is it just an average cost of that? You've already
kind of touched on how folks can modify that. But I think, "What's the default
amount based on?"
Rosanna Coffey: The default amount in the
baseline is the average for the population that we're talking about. So if
we go back to the calculator, this baseline cost of $5,705 is for Medicaid.
It's for children, because I had checked children. And it's for those who had
persistent asthma with an acute visit. Those are the things I have selected,
and it is an average of paid amount to the hospital for people in that population.
Does that help?
Jeff Brady: Yes. I think so.
Rosanna Coffey: O.k. Let me see where I
was. I think we're ready to move on.
I was going to take you to the program impact page. This is where we have
all of the information summarized from the literature. We went through 52 studies,
we broke them down, and we captured information on all of the characteristics
of those studies. And we noted whether they were randomized controlled studies,
statistically controlled studies, or studies with no control group. There are
all types of controlled studies, including the randomly controlled studies,
as well as the statistically controlled studies. We put both in here because
there are so many more studies that are statistically controlled, and that
increases the precision of our estimates.
But we included the randomized controlled studies as a separate option, because
if you're trying to convince physicians that this is going to be a good thing
to do, they are going to want to hear about randomized controlled studies.
The interesting thing here is that the numbers change very little. The results
were very similar for the randomized and statistically controlled studies in
our work. When I go to studies with no control group, the estimates change
a lot, and this is the problem of regression to the mean, because the people
that were high cost will come down in cost anyway over time.
We only include this no-control group in here, because if you're doing a pilot
study, and you've got some initial results, and you don't have a control group
fully analyzed at some point, and you want to compare your pilot study results,
you can compare them to these studies with no control groups. But I think it
is so powerful to see that there is a big difference here between the controlled
and uncontrolled results, that you'll realize that you really do want a control
group when you're doing your evaluation.
Back to our slides. This slide tells you that if you're going to enter your
own data for your pilot test, or you're going to use some study that's more
recent than we have, you're going to have to do the calculation of the percent
change in visits.
You'll do that differently, depending on the type of study you have. If it's
no control group, you'll just do this simple before-and-after percent change.
If it's randomized, you can simply look at the treatment and the control group
I have a little bit of information here about the studies behind the evidence
base. There were 52 studies. They don't all look at all the components of cost.
That's why it's important to pull them all together and synthesize it. These
are in order by the components that have the most evidence behind them. The
ED visits were the most frequent type of study, followed by hospitalizations,
outpatient visits, and so on.
Look at the medication costs. We've only had 10 studies that actually looked
at medication costs. You have to moderate your excitement about those numbers
and what they mean.
Ancillary costs had only three studies. So in those cases, we didn't actually
do a meta-analysis, but we included the average cost effects.
Now we're turning to the program costs, and the design features of the program.
The data fields are the annual costs per participant of the program. This,
you might get from a disease management vendor, or you might tally it up for
your own organization if you're going to incur the cost yourself.
For the number of years until the program has full impact, you decide that.
Also, you'll choose the discount rate and duration of the program. If you're
looking at a program for 3 years, you would specify that, and we can show you
quickly the program costs. It's a simple spreadsheet.
Here is where you put in the annual program costs for participant: $395 is
the average that we found in the literature. It may be high, but at a minimum,
you have a place to enter an amount you may be negotiating with a vendor and
see what the implications are of that particular cost.
Now I will exit the desktop and go back to the slides.
How do you choose a discount rate? What we used basically was the rate of
inflation. It really is just a method to equalize that stream of costs that
you incur over the time of the program and the stream of benefits that you
expect. The cost can be lumped up front, the benefits can be lumped at the
end, and so you're trying to equalize that time differential, and the value
of time and money spent or received is different across time periods. Money
is much more valuable when you've got it in your pocket than when you're waiting
for it out into the future.
You may want to think about how you would estimate the annual cost per participant.
I think here, you may want to call some disease management vendors and try
to get some costs. We tried to do that. We weren't very successful. We didn't
have a real program that we were trying to set up with them, so they weren't
too interested in helping us out on this.
This wraps up the types of data needed. We've talked about all four of these
types of data. When putting it together, I just want to say that it's ideal
if you get it from a single source. It's not essential that you have it from
a single source; you may get one thing from your Medicaid claims, you may get
something else from another study in the literature or your pilot study.
The important thing is that you make sure you're talking about the same population
of asthma severity, Medicaid, privately insured, or children/adults, and that
all of those definitions are the same in the underlying data that you're bringing
into the calculator.
I want to jump to the desktop and show you the results of the calculator.
This is the results page. It has the results related to all of the things
that you've selected in the prior sheets. On the bottom right is the overall
impact. The net present value is just the difference between the total program
costs that you'd see here on the left and the total health care savings per
participant that you see at the top. In this case we did not include productivity
gains, so we have a positive net present value for this population. The return
on investment means that for every dollar that you invest, you're getting $1.05
back. So you are definitely covering the costs of your program. In this case,
if it costs $395.
One thing you do have to do is hit the Update button if you need to refresh
those. Even though the estimate of costs we had in here was $395, you can see
from here, you can go up to a program cost of $412 and still cover the costs
of that program in what we expect to return.
What if I had selected a population that was persistent asthma, not just those
with an acute visit? Say, I was trying to reach a higher number of people.
So when I go to the results page, I don't have a positive net present value
here. I get, for every dollar I spend on all people with persistent asthma,
only 32 cents back. Thus, I'm going to be subsidizing the improvement and care
for these people.
By playing around you will see where you will get a return and where you will
not get a return but will be eating some of the costs of the program. If we
could get our costs down to $126 for the cost of this program, we would be
just covering our costs. If I included productivity gains, I could go up to
$213 for the cost of the program and cover the cost.
So that shows you how the calculator works. And how you can put your own data
into it. Shall we stop for questions? Jeff, do we have questions?
Jeff Brady: I've actually answered a few
questions offline that maybe we should open it up for questions verbally now.
Rosanna Coffey: O.k.
Jeff Brady: Does anybody have questions
Rosanna Coffey: Do we have a question on
the line? Identify yourself and let us hear your question.
Jeff Brady: Sounds like there are not any
questions. I think it was somebody just some offline discussions.
Rosanna Coffey: So, for those of you who
are having discussions and we are listening to them, you may want to mute your
phone so we don't hear them. Shall we move on?
Jeff Brady: Yeah, I think so.
Rosanna Coffey: O.k.
Now we're going to turn to interpretation of the results. I'm going to give
you a little sense of how the results change with different populations. To
start with, I have some default decisions that I made, and I'm going to do
this in the slide, and show you some tables of different scenarios.
I selected the population nationwide. I selected a program that lasts for
5 years. I want to evaluate it within and at the end of 5 years. I assume that
it will take me a couple of years to get my participation up to where I assume
it should be. And the discount rate is 3 percent. You may want to up that,
with all the talk about inflation that's going on. But we did this earlier,
and we used the 3 percent rate. I'm going to use the evidence from randomized
controlled studies, because I have to convince physicians about this.
Here are four scenarios. I have four more on the next slide, which I'll show
you in a minute. In the first three, we're looking at the Medicaid population.
The first scenario is for adults and children, or children only. All of these
are populations with persistent asthma. Annual cost of the program in the first
scenario is $395 and in the second as well; then I lowered it to $100 in the
third. For my assumption about which costs I'm worried about, I'm looking at
this from society's point of view. So I'm counting Medicaid's cost as well
as the patient's costs if there are any copays under Medicaid. I'm not going
to include productivity costs. You can see, nationwide, I expect with my assumptions
about participation of 25 percent that I'd have about 2 million people nationwide
in an asthma program, or 850,000 children.
The net present value is negative in the first two cases, and when I lowered
the program cost from $395 to $100, I got a positive net present value of $123
per person. The return on investment is 27 cents that I receive back in addition
to covering my costs in the third scenario. If I had lowered the cost of the
program in the first case to $56, I would have covered my costs, or if I had
lowered it to $126 for the children-only program, I would have covered my costs.
For the employer-sponsored scenario, I have the same assumptions that I had
in the first three scenarios. The difference is that the employer-sponsored
children don't have as high a baseline cost and utilization costs as Medicaid
children do, and so my return on this is not as high as it was for Medicaid-actually
it's a negative net present value for the employer-sponsored group. From each
dollar that I spend, I only get 87 cents back.
Now the next four scenarios are employer sponsored. The first two are for
children (scenarios five and six) for acute visits for persistent asthma. These
are the people that went to the hospital, were admitted, or were treated in
the emergency room and released. Again, we kept their program costs pretty
low and assumed all medical costs for the patient as well as the plan in the
first case and only the plan costs in the second case.
We did not include productivity, but we have here a net present value, because
we're looking at very high cost cases, those with acute visits. We get a big
return on this particular population when we're looking at those with a lot
of utilization in the past year.
Let's look at this last one, because we have such a huge return on this one.
These are adults, they have persistent asthma, and they have $100 program cost
per year. We include not only the plan cost, but also productivity gains. When
you include the benefits to society of having people not miss work, and you
evaluate that, you have a big return on investment.
How do you use the calculator? You use the calculator to forecast the financial
impact of the program that you might design. Keep in mind that financial return
is not the only thing you may be trying to achieve.
And, you want to assess the key assumptions about your proposed program. Make
sure that you know whether you're doing this with children or adults. The calculator
may help you make that decision. What asthma severity are you going to target?
What are the costs of the program? Are you going to consider social costs,
and is productivity something you think you should be counting as a benefit
One of the things you can do with this is figure out if your own assumptions
are reasonable compared to the evidence in the literature.
You can enter your own data. You might have a pilot test that you've done,
and you want to enter that.
You may be talking to vendors about how much a program will cost. You may
want to put that in here and see how it plays out. You may want to use it to
negotiate with a vendor.
Another way you can think about using the calculator is, suppose you already
have a program going on. What kinds of things should you be evaluating and
including in your evaluation of your asthma care program?
We went through that scenario with the New York Department of Health. I don't
know whether anyone is on from New York, who worked with us. They asked for
some technical assistance. We spent 45 minutes on the phone listening to what
it was they were doing and tried to apply the calculator to their situation.
They were looking at a program that was much bigger than asthma. It was a housing
improvement program but they thought that that improvement program should help
reduce asthma symptoms and they were interested in pulling that aspect into
the evaluation. At the end of our call, we decided the calculator itself wasn't
something they needed to put data into and work through. But they found it
really helpful in terms of thinking about the actual components that they should
be looking at. They hadn't thought about looking at emergency department visits
and hospitalizations, counting the medication costs, and things like that.
So they were interested in it for the conceptual framework that it provided
I also want to mention that we got into this calculator because AHRQ had a
program to work on pediatric asthma. The Michigan Pediatric Asthma Coalition
was part of that. I think Betsy is on the call. We worked with Betsy and her
colleagues in the Michigan Department of Health, and they influenced us a lot
in terms of designing this. I understand that they actually used it in making
a decision about a county-level asthma program. They had to enter county-level
data into a calculator, which you can do, and figure out whether it made sense
for them to do an asthma improvement program in a particular county.
We've also had a request for the calculator from Iowa, the Iowa Medicaid Medical
Officer. I don't have details on that, but they were looking at it in terms
of whether they should start an asthma care improvement program.
The last thing that we wanted to cover if we have time, or if there are burning
questions, we can jump to those. But the last-
Jeff Brady: Actually, Rosanna, let me interrupt
for a few questions that we've received since the last question period.
Rosanna Coffey: Sure. Great.
Jeff Brady: One is generally relating to
our plans for updating the tool as more literature becomes available. What
are our plans for that? I can answer that, actually. We don't have specific
plans yet to update the tool that was performed a year ago, and that involved
reviewing the literature for new findings that could be incorporated into the
literature summary that Rosanna provided at the beginning of the session today.
I think a lot of our future plans for that will depend on interest in the tool
and, in fact, use of the tool.
The other big variable is what actually comes out. Of course, if there are
big studies that would have a more significant impact on the input, then that's
something that we'll take into consideration too. That answers that question.
There have been a few questions that have come through about obtaining the
calculator and how to get that. Presently it's available by request to Margie
Shofer. Her e-mail's at the end, which Rosanna will get to.
There was a specific technical question about the 12-month Medical Consumer
Price Index (MCPI) change as a discount rate. I don't know if Rosanna you want
to take that. The question was: Did you decide to use a 12 month MCPI change
as a discount rate?
Rosanna Coffey: The discount rate that we
used was an annual inflation rate from the Consumer Price Index. It wasn't
the medical care component. We're just trying to equalize the cost of money.
Not the medical care cost inflation, so that's why we used the 3 percent rate.
Jeff Brady: O.k. I think that catches us
up. By all means, if anybody has a question that we haven't addressed that
you sent by text, please let us know, and then we can unmute your phone if
you'd like to ask a question that requires maybe a little more explanation.
Otherwise, I guess we're good to go ahead, Rosanna.
Rosanna Coffey: O.k.
Let's go to limitations/caveats. We just want to tell you when the results
are less than robust.
We only had seven studies that reported program costs. They had a very wide
range from $81 to almost $1,000. We put in the average amount. But I think
that's a place where you'd want to be snooping around and finding out where
you can get cost data, what a program will cost, what it looks like and what
it includes. And maybe enter that information directly, rather than using what
we have in there from the literature.
The other studies that we were disappointed in were: Few studies looked at
medication costs. There were 10 of them. They are the more recent studies.
There may be more recent ones now. The baseline asthma medication cost in those
studies really varied by a lot. So that's another place, maybe if you're going
to play around with it. Maybe look at the literature, think about better-cost
Jeff just discussed the next bullet, which is the plan for updates. They want
to hear from you about that. Literature continues to grow and you can monitor
the literature. We did it through March of 2007, so anything more recent won't
be in the calculator.
Baseline data also do become obsolete. I think I showed you that we used 2005
data for most of this. We did inflate it to 2006 dollars. You may want to inflate
it now if you're looking at 2008 dollars.
And those are basically the main limitations of the calculator.
I think the real advantage of this calculator is that it gives you a real
handle on what the literature tells you. I did not go into how we went about
getting these average impact estimates. We have a very detailed paper that
walks through all of that and the analysis and all of the things we controlled.
We were just impressed with the amount of information that was out there and
that we were able to synthesize, and I can't emphasize enough that it's not
an easy thing to do. Even if you have the time to read all that literature,
you can't wrap your head around it and figure out what the average effects
are, unless you do something quantitative like we did in our meta-analysis.
Let's open the line for other questions at this point.
Jeff Brady: Rosanna, can you hear me?
Rosanna Coffey: Yes, I can.
Jeff Brady: There's a question about Medicaid
programs with a majority of members that are enrolled in managed care or full-risk
capitation programs. The question is: Other than the apparent benefit of improved
health outcomes or productivity gains, how can you apply the ROI tool in that
case? How could it be used?
Rosanna Coffey: If you have a population,
if I understand the question correctly, with the majority of your Medicaid
members in a managed care program, the data that we have that underlies the
calculator is based on about 60 percent of Medicaid enrollees in managed care
in the data that we use to populate the calculator.
If your managed enrolled population is about 60 percent, this is probably
a pretty good estimate for you, although if it were convenient to put your
own numbers in here, I would do that. If your managed care enrollment is 100
percent, I would definitely put your own numbers in to see, because your baseline
utilization numbers are going to be different. And you want to take that into
account when you're looking at a return on investment.
If your baseline costs are low, you may not get the same kind of return on
investment that we get out of this with the average effect. It is in here-the
managed care component. It's reflected in these data, but it just depends on
where you are compared to 60 percent coverage, as to whether this is realistic
Jeff Brady: O.k. Some other questions that
have come in. It's fairly short. Where are the payments for physician care
during an inpatient stay or ER visit captured?
Rosanna Coffey: They're going to be captured
in the inpatient stay or the ER visits because in the claims that we get, we
bundled those into that component. If the care were provided in the ER, if
the care were provided in the hospital, it would be bundled in there. If the
care were in the doctor's office, of course it's going to be under the outpatient
Jeff Brady: O.k. Then there's another question
about access to the articles and the literature. That's part of the tool, right,
Rosanna Coffey: In the paper we wrote, we
have the abstracts for all the articles that we included in the study as well
as some that we did not include, because their metrics were not set up so that
we could use them. You could go to our appendix in that paper and look at all
the study abstracts. Then go and obtain the articles online or from the journal
itself. That's how you would get access to it. But you can get a lot of information
out of our paper.
Jeff Brady: O.k. Just to clarify another
question that's come through a couple of times, is this just for an Asthma
ROI Calculator, just for this disease only at this point?
Rosanna Coffey: That's definitely correct.
It's just for asthma. It's the asthma literature that drives the impact estimates.
Asthma was the population we looked at for the baseline estimates, and you
could not use this for, say, diabetes or some other disease. We wouldn't put
our Good Housekeeping Seal on it.
Jeff Brady: Right. O.k. I think that catches
up all the questions that I've seen by text, but it's possible that I've missed
a few. If that's the case and somebody wants to ask their question over the
phone line, that's something we can certainly do now or, additionally, if there's
followup on one of the questions that has already been asked, and some additional
clarification is needed, we can do that at this time, too.
Rosanna Coffey: I think the other thing
we ought to do before we sign off is do another poll and ask the question:
How many organizations out there think they would like to have some technical
assistance on the calculator? So, would you be interested in receiving technical
assistance on the calculator? Just if you wouldn't mind answering that question
at this point, and we'll get some idea.
It could be providing some presentations to a group in your organization,
or it could be working through the calculator with you and your particular
situation, like we did with New York. It could be holding your hand while you
calculate numbers. All kinds of things.
We are willing to listen to your voice if you'd like to ask a question verbally
and don't want to type. That works too for us.
O.k. Would you be interested in technical assistance? Seventy-six percent
said yes. I have to find out how many were under that. Jose, can you tell us
how many respondents there are? I see the percent, but I don't see the number.
We'll figure that out.
We have a little more than 5 minutes left. Who am I trying to get back to,
Margie? Are you on mute, Margie?
Margie Shofer: Hello.
Rosanna Coffey: There you are.
Margie Shofer: Thank you, Rosanna. Thank you, Jeff, for the
presentation today and thank you everyone on the line for your thoughtful questions
and your participation in this Webinar. We hope you found this discussion helpful.
If you have questions about follow-on technical assistance opportunities, please
do not hesitate to contact me, Margie Shofer.
Rosanna Coffey: I have that up on the screen.
Margie Shofer: That's my contact information in the first
bullet on the slide. If you have any questions or comments about the tool,
or would like to request a copy of the calculator, please send an e-mail to
the AHRQ Quality Tools mailbox. That's the second bullet there. For more information
about the suite of AHRQ tools developed for the State Quality Improvement Project,
please visit the site on that final bullet. Under that final Web address, a
bunch of you asked if there was going to be a recording of this.
And there is. We're going to have a recording of events, as well as a transcript
available. I'm not sure how quickly it will be up, but it will be up there
hopefully sometime soon. So I want to thank you again for being on this Webinar.
And this concludes the Webinar and we look forward to hearing from you. Bye.
Rosanna Coffey: Goodbye.
Current as of May 2009
Return to Contents