Skip Navigation U.S. Department of Health and Human Services
Agency for Healthcare Research Quality
Archive print banner

Asthma Return on Investment (ROI) Calculator Web Conference

This information is for reference purposes only. It was current when produced and may now be outdated. Archive material is no longer maintained, and some links may not work. Persons with disabilities having difficulty accessing this information should contact us at: Let us know the nature of the problem, the Web address of what you want, and your contact information.

Please go to for current information.

Questions and Answers

On September 9, 2009, Dr. Rosanna Coffey, Ginger Carls, and Elizabeth Stranges presented the Asthma Return-on-Investment (ROI) Calculator during a Web conference. Stephanie Lindsay from the Alabama Medicaid Agency provided brief comments about her Agency's use of the tool. These are the questions asked by participants during the event, with the associated answers.

Asthma Return on Investment (ROI) Calculator Web Conference: Question and Answer

Question: Is the ROI Calculator free on the AHRQ Web site?
Margie Shofer: It is. It is available at:

Question: What size population do you need to be able to use this Calculator? Are 10,000 patients enough or do you need bigger numbers?
Rosanna Coffey: You can use any size population in this Calculator. It just doesn't make sense to do 1 or 2 patients or 10 patients because you're going to have such variability in small numbers in the estimates that you come up with. But there's no limit on the size of your population.

Question: The major focus of this Asthma Calculator seems to be hospitalizations and ER visits. Does it include anything else?
Rosanna Coffey: It's also outpatient visits, medication costs, and ancillary testing. All of those components are in there.

Question: Do the Medicaid data include SCHIP?
Ginger Smith Carls: The Medicaid data include several States. I think it's about 8 to 10 and those States, exactly how they're organized, the way they cover children, I'm not sure, but it includes all their populations. So basically, we got our baseline cost and utilization data from those 8 to 10 Medicaid States. For prevalence, we have that from some mix from Medicaid services, so that would include the SCHIP programs. Does that answer the question?
Rosanna Coffey: Ginger, are you saying that SCHIP was included in some of the Medicaid States that we have and maybe not all? And that's something we would have to look up to know?
Ginger Smith Carls: I think we have to look up to know for sure. We didn't exclude anyone, so whoever is covered by their State's Medicaid program would be in there.

Question: Two questions about the meta-analysis: Is it published? And could the list of the 50 studies that were used be provided; specifically, in what kind of settings were these done, and how does the intensity of the intervention vary in different studies?
Rosanna Coffey: To the first question: the study has not been published. For the second part, we can provide a list. We have documentation on the meta-analysis, the information on all the studies and their abstracts. That information is available. Please e-mail me ( or Ginger ( for that information.

Question: How can I find out about the asthma programs used?
Rosanna Coffey: That's also in our documentation and the abstracts of the studies will give you some information about that. We have that readily available. But if you want more detail, you have to go back to the actual studies.

Question: Were acuity levels considered in the ROI Calculator methodology - self-management versus clinical touch?
Rosanna Coffey: I'm not sure what you mean by self-management versus clinical touch. But the ROI Calculator distinguishes between anyone with asthma and people with persistent asthma and an acute visit in the year, which means they had a hospital stay for asthma or an emergency department visit for asthma.
The studies were classified based on the classification of severity in terms of how they described their sample and also based on their data. If they recruited everyone in their study from the ER, then they were in the persistent asthma with an acute visit category. A lot of the studies described their population as everyone with asthma and this is a wide range of severity. Or they said they were looking for the highest cost cases or the more persistent, more complicated cases.

Question: Does the Calculator offer a 95 percent confidence interval around the ROI estimates that it gives?
Ginger Smith Carls: No, it does not.

Question: For medication costs, does it calculate medication costs to the patients?
Rosanna Coffey: I think what they're asking is, there are multiple payers for medication costs. There are the patients paying their copays or deductibles, there is the insurance, possibly, or if it's Medicaid, then Medicaid pretty much pays for all of it. The way we have set up the Calculator, you can select all the payments made or just the payments made by either Medicaid or a government employer or the insurance company. So you could look at it from a cost perspective. That's a great question.

Question: Can the ROI Calculator be refined to a stratification level, high, medium, low level of asthma severity and from the stratification help to identify the most cost-effective approach to asthma management?
Rosanna Coffey: Yes.

Question: Could you please elaborate on the cost of the asthma programs and how you arrived at the cost for the programs?
Ginger Smith Carls: That's a great question. Only a few of the studies actually published the cost for them to implement the program. There were only seven, so we just took the average of those seven studies. There are very limited data related to cost.

Question: My question is about asthma severity. My guess was that you have categorized it as such based on what was available, but obviously this is not completely in line with the National Asthma Education and Prevention Program (NAEPP) guidelines insomuch as you're only looking at persistent asthma. But obviously under the persistent category, there's no breakout of mild, moderate, severe. Is that something that you're able to elicit from the studies that were published and can add into the Calculator at a later time?
Ginger Smith Carls: I was involved in reviewing the literature. We did look at how finely we could break that out and, based on what those studies reported, it seemed like all you could really get out of it was those levels that we used. Because there were variations of how detailed they described their populations. So that would be a great improvement. But certainly no literature through 2007 really supported that level of detail. There may be some new studies that fine tune it more.
The other piece of that, not just the literature, is that you want to collect information on what it costs to treat mild but persistent asthma. We decided to use medical claims data. We figured other users of the Calculator might want to use something similar. I don't believe that you can get that fine a resolution from claims data. Maybe I'm wrong on that, maybe you know more about that, but I think it's just we use these data. The persistent asthma definition is based on the Healthcare Effectiveness Data and Information Set (HEDIS) definition and it's based on medication use and also utilization of hospital visits, ER visits, and any excessive outpatient visits.

Question: What are your recommendations on how to determine the impact of a program?
Rosanna Coffey: I assume you're talking about the impact of the program that you would implement. What you're getting out of the Calculator is an assessment of your population and perhaps your baseline utilization or cost information compared against what the literature would tell you is the average impact that you would expect for the particular one selected. So you're getting a view of what others have accomplished. And then what you would want to do is look at your own data, do a before/after and a control group and look at the change in all of those components that are in the Calculator and calculate a utilization change. Then estimate the costs - as we've done in the Calculator - and come up with your own program impact for the program that you've implemented. It requires more work than just going in and putting things into the Calculator because there you're seeing what other people have achieved. You want to know what your program has achieved. But you can follow the same methods to figure it out for your program.

Question: Could you please explain break-even program impact again in the results section?
Ginger Smith Carls: There's a field called Break-Even Program Impact and that tells you how much you need to reduce - and I believe it's in percentage terms - the cost to get to break-even, meaning for every dollar you invest you get a dollar back. There's also a break-even program cost that tells you, given whatever you put in for a pilot study, how much your program should cost be so that every dollar you invest on the program will yield a dollar in savings.
So that would be the number you'd want to take to your vendor and negotiate with to try to achieve.

Question: Can we use the ROI Calculator to estimate long-term impact after the program has ended? For example, if patients get rid of triggers for asthma or know how to manage their asthma for the rest of their life.
Rosanna Coffey: That's a good question. The studies that we have - and Ginger, you're going to want to jump in probably - the studies that we have for the most part are 2-year studies.
Ginger Smith Carls: They're mostly 1-year studies. There are a few that are longer, but these longer term impacts are not really well studied. It would be great to have that information but certainly in the literature we looked at, it's not there. Maybe there's some more information out there since the enterprise, but there's not a whole lot of evidence on that, unfortunately.

Question: How can you get estimates for the average ED visit per patient?
Rosanna Coffey: It depends on what population you're looking at. For Medicaid, they certainly have that information; health insurance carriers, private insurance companies would have that information. That's where most of this information comes from.

Question: How might the ROI Calculator allow an asthma practice to document higher quality of care for asthma patients?
Rosanna Coffey: This is a very interesting question because you're trying to get down to the individual physician or at least the clinic level in terms of asthma practice. The ROI Calculator is designed for a look at a population. If you have a substantial population that you're tracking, you could certainly look at the Calculator, play with it, see all the components I mentioned early in the slide set, and evaluate those for your particular population. If you have a clinic that can poll information on people with asthma, we have in the Calculator definitions of what persistent asthma is for this purpose and persistent asthma with acute events. If you can go back into your population and calculate those things, you can enter those and evaluate where you have room to improve. In terms of delivering higher quality of care for asthma patients, certainly you know that the NAEPP is basically what most of the literature followed in setting up asthma care improvement programs. So that's what you're being measured against in this Calculator. If your clinic already subscribes to all of those activities - the written asthma plan and education for the patient and encouraging the patient to take responsibility for managing the asthma and the controller medications and all of those things - then maybe you're already there. But by looking at all of the information behind the Calculator, you would maybe be able to figure that out in terms of where you need to improve.

Question: In a private practice allergy and asthma group, we aren't privy to the number of primary care or ER visits they have had. We would have to work with a big insurance company to access the data. Are the insurance companies willing to provide the data to a smaller allergy and asthma practice?
Rosanna Coffey: That's very good question. I can't answer it for the insurance plans that you're working with. I would think that they would be interested in improving the care for the patients that they're insuring, but I can't really know what they would do. I appreciate what you're saying. You don't know the number of ER visits; you don't know the total number of outpatient visits. But you could look at the baseline data and try to look at how your population compares for the thing that you do know - medication costs perhaps - but you may even have trouble with that. So the Calculator was really built for a big population kind of view. And trying to decide even on a State level, whether to undertake and invest in putting people into some kind of a care management program.

Question: It still seems like cost of medications and other utilization numbers are critical. Where do we find these numbers: carriers, Medicaid, etc.? Where we collect this cost seems confusing with multiple carriers. Would this work for a moderate-size practice?
Rosanna Coffey: Again, this is for a practice and trying to find these numbers for your patients, I agree with you, is going to be difficult. I don't know whether you have any ideas, Ginger.
Ginger Smith Carls: The problem here is, again, the practice is just one part of the system and this requires an integrated system to be able to look at costs from all these different perspectives, not just outpatient but inpatient. So I don't know where you would get that. You could use what we have as an approximation. But certainly you'd probably want something more custom if you were going to use it in some sort of negotiations. I'm trying to think whether, if you specify your population more carefully and then go to a carrier, or even a Market Scan database, we could tease out estimates for use for a more careful definition of your population. But that's a synthetic estimate for you

Question: Will this presentation will be available for later review online?
Margie Shofer: Yes. We will have it up on the AHRQ Web site later but sometimes it takes us a little while to get these up, so please be patient. We probably will send out an E-mail letting people know when it is available.

Return to Contents

Current as of October 2009

The information on this page is archived and provided for reference purposes only.


AHRQ Advancing Excellence in Health Care