Skip Navigation U.S. Department of Health and Human Services
Agency for Healthcare Research Quality
Archive print banner

Hands-on-Training: Screen shots of the asthma care return on investment calculator

This information is for reference purposes only. It was current when produced and may now be outdated. Archive material is no longer maintained, and some links may not work. Persons with disabilities having difficulty accessing this information should contact us at: Let us know the nature of the problem, the Web address of what you want, and your contact information.

Please go to for current information.

On December 6, 2007, a panel composed of Ginger Smith Carls, Rosanna M. Coffey, Ronald J. Ozminkowski, Karen Ho, Mika Nagamine, and Katheryn Ryan made a presentation entitled Hands-on-Training: Screen shots of the asthma care return on investment calculator. This is the text version of the event's slide presentation. Please select the following link to access the slides: (PowerPoint® File, 1.1 MB).

Slides: 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22

Slide 1: Hands-on-Training: Screen shots of the asthma care return on investment calculator

Ginger Smith Carls, M.A., Rosanna M. Coffey, Ph.D. , Ronald J. Ozminkowski, Ph.D., Karen Ho, M.S., Mika Nagamine, Ph.D., Katheryn Ryan

December 6-7, 2007
State Healthcare Quality Improvement Workshop:
Tools You Can Use to Make a Difference

On the top of the slide are the logos for the Department of Health & Human Services and the AHRQ logo. The Department of Health & Human Services logo is an artistic image of an eagle with the outlined profile of faces. The AHRQ logo reads, "AHRQ — Agency for Healthcare Research and Quality: Advancing Excellence in Health Care,"

Return to Top

Slide 2: Steps in the ROI calculator

This presentation uses a template with a blue background and a header with the AHRQ and Department of Health & Human Services logos on the left. The header and body of the slide are separated by a light blue horizontal line that traverses 80 percent of the slide from the left.

This slide shows a series of steps in the ROI calculator. The steps include:

  1. Describe population
  2. Estimate number of participants
  3. Estimate baseline utilization or missed work days
  4. Estimate impact of the asthma program
  5. Estimate program cost

In all five steps, the user chooses between default data and user data. Meta-analysis occurs at steps 4 and 5.

Return to Top

Slide 3: Data Sources

  1. Demographics of populations.
      — Medicaid (CMS 2003).
      — Employer sponsored health insurance (CPS 2003-2005).
        — State employees (Employees from BLS 2003-2005).
  2. Large, nationwide, medical claims database (MarketScan™).
      — Prevalence rates.
      — Utilization and costs for asthma patients.
  3. Literature review.
      — Impact of asthma care programs.
      — Cost to implement asthma care programs.

Return to Top

Slide 4: Example

  • Based on default values of the calculator when calculator is opened.
  • Asthma care program for children and adults with persistent asthma for Medicaid programs (nationwide).

Return to Top

Slide 5: Population

This slide contains a screen shot of the Population page from AHRQ's Asthma Care ROI calculator. Users are prompted to select the type and geography of the general population (people with and without asthma).

Return to Top

Slide 6: Participants

This slide contains a screen shot of the Participants page from AHRQ's Asthma Care ROI calculator. The screen prompts the user to select the age group and asthma severity to target for a quality improvement program. Next, the user is to review the number of people eligible for the program and the percentage expected to participate in the program.

Return to Top

Slide 7: Asthma Definitions

  • All Asthma
      — Patient has at least one claim with asthma as a primary or secondary diagnosis during the year (ICD-9-CM code 493.xx).
  • Persistent asthma*
      — One or more of the following is true using one year of data:
    1. ED visit or inpatient admission with asthma diagnosis.
    2. 4 outpatient visits and at least 2 asthma medication fills.
    3. 4 asthma medication fills, if all 4 fills for leukotrienes, then must have at least 1 claim with an asthma diagnosis.
  • Persistent asthma* with acute visit.
      — Met criteria a) for persistent asthma.
  • Similar to HEDIS definition, differs in that only 1 year of data is used.

Asthma medications include: Antiasthmatic combinations, Bronchodilator combinations, Inhaled anticholinergics, Inhaled corticosteroids, Leukotriene modifiers, Long-acting adrenergic bronchodilators, Mast cell stabilizers, Methylxanthines, Short-acting adrenergic bronchodilators, Corticosteroid tablets or syrup (oral corticosteroid).

Return to Top

Slide 8: Baseline Data

This slide contains a screen shot of the Baseline Data page from AHRQ's Asthma Care ROI calculator. Two decisions must be made on the page in order to use: 1) Select a treatment definition and cost perspective for calculating results, and 2) Review the baseline utilization and cost estimates.

Return to Top

Slide 9: Program Impact

This slide contains a screen shot of the Program Impact page from AHRQ's Asthma Care ROI calculator. The page shows the impact estimates from a meta-analysis of the research literature on asthma quality improvement and disease management. Both health care measures, such as program impact on emergency department visits, and productivity measures, such as program impact on missed work days per adult, can be assessed using this tool.

Return to Top

Slide 10: Program Cost

This slide contains a screen shot of the Program Cost page from AHRQ's Asthma Care ROI calculator. The page shows four decisions that will affect calculations of the cost of the program to be implemented:

  1. Length of operation planned for the program.
  2. Time until the full-impact of program is expected.
  3. Cost of the program per person per year (consider changing based on costs estimates for specific programs from vendors).
  4. Discount rate for valuing savings and costs that occur in different years.

Return to Top

Slide 11: Results

This slide contains a screen shot of the Results page from AHRQ's Asthma Care ROI calculator. The page shows the impact of the asthma program and summarizes prior assumptions made in the calculator. Two decisions must be made on this page: 1) Whether to include health care savings, productivity gains, or both. 2) Whether to report results per participant or per program.

Return to Top

Slide 12: Undiscounted Results per Person for the User-Specified Program Tabulated by Outcome Measure and Year

This slide contains a screen shot of the Undiscounted Results per Person for the User-Specified Program page from AHRQ's Asthma Care ROI calculator. The page shows the annual utilization and cost per patient for emergency department visits, hospital confinements, outpatient visits, emergency department cost, hospital confinement cost, outpatient cost, asthma-specific ancillary cost, asthma medication cost, missed work days for adults, missed school days for children, productivity cost for adults, and productivity cost for caregivers. The data are shown for sample years 2007 to 2012. Data for the earliest year represent the baseline period before the program is implemented.

Return to Top

Slide 13: Literature Review Methods

  • Inclusion criteria
  • Characteristics of included studies
  • Calculation of outcome (impact of the program)
  • Analysis

Return to Top

Slide 14: Study Inclusion Criteria

  • Studies searched (76 studies met criteria):
      — Asthma care quality improvement: Physician & patient training—written asthma action plan, etc. (not drug efficacy).
      — U.S. studies: 1995 thru 2006.
      — Populations: Children and adults under 65.
      — Studies: Individual interventions (not meta-analyses or editorials).
      — Impact period: 6+ months.
      — Impact: Use or cost of medical care services and productivity (not asthma knowledge or quality of life).
      — Insurance: All types.

Return to Top

Slide 15: Abstracting literature

  • Details recorded.
      — Characteristics of studies and patients (e.g., asthma severity).
      — Baseline utilization and cost by patients in the studies.
      — Impact of program on percent change in visits or days (for default values).

Return to Top

Slide 16: Included studies

  • The 52 studies report 261 pieces of information.
      — Related to service utilization and productivity.
      — For children, adults or both.
      — With multiple items (e.g., on ED visits, if the study reported results separately for children and adults).
  • Outcomes (number of related study results)
      — Emergency department visits (90)
      — Hospital stays (75)
      — Outpatient visits (57)
      — Medication costs (12)
      — Cost of ancillary services (3)
      — Missed work or school days due to asthma (24)

Return to Top

Slide 17: Study Characteristics

  • Useable means study had to report average visit rates, missed days, or cost (not just percent with event).
  • Most useable studies:
      — Had a control group (56%); many had randomized controls (38%).
      — Involved patient self-management (85%) and regular medical management (58%); many had a written asthma action plan (40%).
      — Focused on children only (46%).
      — Focused on populations with persistent asthma (55%).
    • Persistent asthma = subjects met HEDIS criteria OR had 2 or more hospital or ED visits at baseline.
       — Had mixed insurance coverage (79%).

Return to Top

Slide 18: Included Studies and Patients (includes patients in treatment and control groups)


Cost components Children only Adults only Both Total
  Patients Studies Patients Studies Patients Studies Patients Studies
ED Visits 13,213 21 714 11 8,812 9 22,739 40
Hospitalizations 17,575 19 7,161 9 2,526 7 27,262 33
Outpatient visits 20,229 18 6,986 4 1,888 5 29,103 25
Missed work/school days 4,172 11 521 5 443 3 5,136 17
Medication cost 486 2 301 3 13,580 5 14,367 10
Ancillary service cost 61 1 148 2 0 0 209 3
Total direct cost 2,729 8 649 4 10,131 8 13,509 20
Total indirect cost 0 0 143 2 524 2 667 4

These data indicate that more patients and studies on children are included, with the most patients and studies on children only, studying ED visits, hospitalizations, and outpatient visits.

Return to Top

Slide 19: Study Designs

  • Randomized controlled trial (RCT) studies.
      — Standard for clinical efficacy.
      — More likely to be accepted by clinicians—main target of QI programs.
  • Statistically controlled studies.
      — Much larger samples with greater precision.
      — Can control for more patient and setting attributes than RCTs.
  • Studies without a control group.
      — Included for showing importance of the study design in measuring outcomes accurately.
      — For a preliminary benchmark that can guide a program in its early days about its success or failure.

NOTE: Results are available for each design type.

Return to Top

Slide 20: Program impact calculations

  • All studies.
      — Pre-post treatment comparison.
  • Controlled
      — Post treatment-to-control comparison.
      — Net change pre-post, treatment-to-control comparison.
  • Example calculation.
Data ED visits per person per year
Before intervention After intervention
Treatment group 4 2
Control group 4 3
Pre-post treatment: (2-4) / 4*100 = -50%
Post treatment-to-control: (2-3) / 3*100 = -33%
Net change: [(2-4) / 4*100] - [(3-4) / 4*100] = -50% - (-25%) = -25%

Return to Top

Slide 21: Analysis of Results

Regressions: study-result outcomes regressed on study population and design features.
  — Ys: ED visits, hospitalizations, outpatient visits ,missed work/school days, and medication costs.
  — Xs: Study population, study design, sample size, length of study, and contact with the physician or patient.
  — Bs: Average impact of each study feature on Ys, controlling for other Xs.

Other issues:
  — Standard errors adjusted for multiple results per study.
  — Studies weighted equally, by using inverse of number of results per study as weight on each study-result observation.
  — Ancillary services not analyzed in regression context, due to the small number of studies.

Return to Top

Slide 22: Selected literature reviews

Bernard-Bonnin, A.-C., S. Stachenko, D. Bonin, C. Charette, and E. Rousseau. 1995. "Self-management teaching programs and morbidity of pediatric asthma: A meta-analysis." J Allergy Clin Immunol 95(1):34.

Krause, D. D. 2005. "Economic Effectiveness of Disease Management Programs: A Meta-Analysis." Disease Management 8(2):114-34.

Lee, T. A. and K. B. Weiss. 2002. "An update on the health economics of asthma and allergy." Current Opinion in Allergy and Clinical Immunology 2(3):195-200.

Sullivan, S. D. and K. Weiss. 2001. "Health economics of asthma and rhinitis. II. Assessing the value of interventions." Current reviews of allergy and clinical immunology 107(2):203-10.

Shojania KG, McDonald KM, Wachter RM, Owens DK, eds. 2007. "Closing the Quality Gap: A Critical Analysis of Quality Improvement Strategies/ Vol 5: Asthma Care." Technical Review 9 (AHRQ 04(07)-0051-5).

Smith, J. R., M. Mugford, R. Holland, B. Candy, M. J. Noble, B. D. W. Harrison, M. Koutantji, C. Upton, and J. Smith. 2005. "A systematic review to examine the impact of psycho-educational interventions on health outcomes and costs in adults and children with difficult asthma." Health Technology Assessment 9(23):1

Return to Top

Return to Contents

The information on this page is archived and provided for reference purposes only.


AHRQ Advancing Excellence in Health Care