Skip Navigation Archive: U.S. Department of Health and Human Services U.S. Department of Health and Human Services
Archive: Agency for Healthcare Research Quality
Archival print banner

This information is for reference purposes only. It was current when produced and may now be outdated. Archive material is no longer maintained, and some links may not work. Persons with disabilities having difficulty accessing this information should contact us at: Let us know the nature of the problem, the Web address of what you want, and your contact information.

Please go to for current information.

Health Information Technology and Patient Outcomes: AHRQ Sponsored Evidence and Next Steps (Text Version)

Slide presentation from the AHRQ 2010 conference.

On September 28, 2010, Stephan Parente made this presentation at the 2010 Annual Conference. Select to access the PowerPoint® presentation (695 KB).

Slide 1

Health Information Technology and Patient Outcomes: AHRQ Sponsored Evidence and Next Steps

Health Information Technology and Patient Outcomes

AHRQ Sponsored Evidence and Next Steps

Stephen T. Parente, Ph.D., University of Minnesota
Jeffrey McCullough, Ph.D., University of Minnesota
Jean Abraham, Ph.D., University of Minnesota
Martin S. Gaynor, Ph.D., Carnegie Mellon University

September 28, 2010

Carlson School of Management, University of Minnesota

Slide 2

Presentation Overview

Presentation Overview

  • Policy & industry context for research.
  • Data & empirical methods.
  • Results & interpretation.
  • Policy Prescription for more HIT and better (or at least more) clinical effectiveness data.

Slide 3

What do We Know about National IT Impact Measured by Data in the US

What do We Know about National IT Impact Measured by Data in the US

  • Actually, very little.
  • Studies generally extrapolate from case examples in a set of national sites.
  • Very little mainstream health insurer success stories.

Slide 4

The Issue With Regional Insurer EMR Cases Applied to the Nation . . .

The Issue With Regional Insurer EMR Cases Applied to the Nation.

  • Hi, I'm a PPO design and I have 85%+ market. I also rule the FEHBP market and TRICARE.
  • Hi, I'm a HMO design and I have -15% market. Oh, and I'm the model for EMR success stories in Colorado, West Coast, North Central PA and Massachusetts. I'm so ACO ready!!!

Image: Photographs of the two men who represent Apple and Microsoft in the commercials for Apple.

Slide 5

What if No One Wants a Trojan Rabbit?

What if No One Wants a Trojan Rabbit?

Image: A still from Monty Python and the Holy Grail, featuring King Arthur and his knights standing in front of the wooden "Trojan Rabbit."

  • Sir Bedivere the Wise: "Now once we have gotten all the physicians to buy a Stimulus Bill-financed Dell computers from Wal Mart for an EMR install, we can distribute the software to them to place more data entry onto their existing workflow and then pay them less when we use the system to tell them they are under-performing in their new ACO/medical home."

Bonus Film Points: Also from 1975: Chouinard A. Shall I not ask for whom the [electronic] medical record is kept? CMAJ 1975. Start of SNOMED

Slide 6

Conceptual Model

Conceptual Model

  • The conceptual model for our proposed analysis is an economic model of technical production.
  • We assume hospitals produce a number of different outputs: quantity (Q) and quality (Z) subject to the following technical production relation, as F (Q, Z, L, K, IT; θ, τ, ε) = 0.
    • L: Labor
    • K: Capital
    • IT: IT Systems
    • θ Patient attributes affecting efficiency (e.g., severity)
    • τ Hospital specific factors
    • 1st derivatives with respect to outputs are positive (FQ, FZ > 0)
    • 1st derivatives with respect to inputs are negative (FL, FK, FIT < 0)
    • 2nd derivatives with respect to inputs are positive.

Slide 7

Data for Empirical Investigation

Data for Empirical Investigation

  • We measure HIT value by combining hospital- and patient-level data during 1997-2007.
  • Sources:
    • Medicare inpatient admissions during our study period—the 100% MedPar inpatient Medicare claims data file. These data provide patient-specific outcomes and severity adjustment measures.
    • The Healthcare Information and Management Systems Society (HIMSS) Analytics Database provides detailed hospital IT adoption data for a variety of applications including:
      • Electronic medical records (EMR),
      • Nurse charts, and
      • Picture archiving communications systems (PACS).
    • HIMSS Analytics comprises a near census of acute care, urban, nonfederal US hospitals.
    • American Hospital Association's (AHA's) annual survey which describes hospital characteristics.

Slide 8

Econometric Approach - 1

Econometric Approach – 1

  • We regress patient-level PSIs on a set of HIT variables, patient-level controls, and hospital fixed effects.
  • Each of the PSIs is a binary variable equal to 1 if an adverse event occurred and zero otherwise.
  • Control variables include patient age, gender (female=1, else=0), race (non-white=1, else 0), risk score, and year of admission.
  • HIT variables are a set of three binary indicators for the presence of EMR, nurse charting, and PACS.
  • HIT variables were lagged by one year to reflect anecdotal evidence and expert interviews indicating that HIT value is realized one or more years subsequent to adoption.

Slide 9

Econometric Approach - 2

Econometric Approach – 2

  • HIT value may change with time through unobserved learning and innovation:
    • We include a set of nine HIT-by-year interaction terms allowing HIT to have a different affect in each year.
    • Interactions of the binary HIT application variables with binary indicators for the years 2000, 2001, and 2002 respectively.
  • Finally, we control for unobserved hospital attributes by including hospital-specific fixed effects.
    • Creates over 2,700 binary variables, one for each hospital in the study. These fixed effects control for hospital attributes that are stable across time such as bed size and patient case load described.
    • This design controls for unobserved time-invariant quality differences. Effectively, this specification controls for some types of selection in the HIT adoption process.

Slide 10

Descriptive Statistics

Descriptive Statistics

Hospital descriptive statistics, 1999 values

Variables BY EMR adoption
Sample Average Early adopters Late adopters Non-adopters
Bedsize 187 233*** 197 179
Visits 129,619 173,272** 136,549 122,467
COTH Membership 8% 18%*** 8% 7%
Multihosp. System 69% 67% 72% 68%
Nonprofit 71% 87%*** 71% 68%**
For-profit 15% 1%*** 15% 16%
Government 15% 12%** 14%* 15%
% Medicare 47% 46% 46% 47%
% Medicaid 18% 17% 18% 18%
Number 2,846 247 522 2,077

* denotes significance at p=0.10, ** at p=0.05, and *** at p=0.001

Slide 11

Health IT Adoption

Health IT Adoption

Image: A line graph titled "Figure 1: HIT Application Diffusion" displays the following data:

Year EMR PACS Nurse Chart
1999 9% 2% 59%
2000 10% 8% 67%
2001 10% 9% 69%
2002 28% 19% 72%

Slide 12

Descriptive Statistics of Medicare Hospital Admissions in 1999, by type of IT Invested

Descriptive Stats of Medicare Hospital Admissions in 1999, by type of IT Invested

  Electronic Medical Records Imaging PACS Nursing IT Systems No EMR/PACS Nursing IT
Patient Attributes
Age 73.4 73.4 73.8 74.0
Female Gender (%) 56.5% 57.2% 57.2% 57.8%
Non-White (%) 15.6% 10.8% 15.2% 16.4%
Risk Score 2.39 2.38 2.30 2.18
Patient Safety Indicator (PSI) admission rate per 1,000 patients
Aggregate PSI 8.628 8.407 7.831 6.794
Complications of Anesthesia 0.191 0.135 0.244 0.204
Infection due to Medical Care 3.061 2.893 2.804 2.389
Post-operative Hemorrhage or Hematoma 2.665 3.006 2.498 2.415
Postoperative Pulmonary Embolism or Deep Vein Thrombosis 13.547 12.768 12.812 12.708
Post-operative Wound Dehiscence 4.201 3.499 4.011 3.876
Hospital Attributes of Patients Admitted
Number of Hospital Beds 368.5 340.0 331.6 293.4
Number of Nurses on Staff 575.3 515.0 469.2 405.4
% Medicaid Admissions 13.3% 13.2% 13.9% 15.9%
% Medicare Admissions 47.7% 47.8% 49.3% 48.9%
Admissions in 1999 879,723 167,115 5,118,437 2,375,527

Based on all Medicare patients treated at 2,707 US hospitals.

Slide 13

Change in Patient Safety by Technology

Change in Patient Safety by Technology

Change in Patient Safety by Technology (per 1,000 admissions)

  All Years
Electronic Medical Records
Infection due to Medical Care -0.490*
Post-operative Hemorrhage or Hematoma -0.240*
Postoperative Pulmonary Embolism or DVT -1.000*
Infection due to Medical Care -0.610*
Post-operative Hemorrhage or Hematoma -0.072
Postoperative Pulmonary Embolism or DVT -2.000*
Clinical/Nursing IT
Infection due to Medical Care -0.390*
Post-operative Hemorrhage or Hematoma -0.009
Postoperative Pulmonary Embolism or DVT 0.300

Notes: * denotes a reduction/increase in clinical error with t-statistic significant at p<.001

Slide 14

IT Total and Year-Specfic Effects

IT Total and Year-Specific Effects

Year-specific and system-specific effect of Health IT on patient safety indicator, adjusted by patient risk and hospital specific attributes (per 1,000 admission)

  IT in all Years Year-specific IT Impact
2000 2001 2002
Infection due to Medical Care
Electronic Medical Records -0.2912** -0.0134** -0.0391** -0.0685**
PACS -0.3056 -0.1010 0.0533 0.0512
Clinical/Nursing IT -0.0053 -0.0331 -0.0632 0.1479**
Post-operative Hemorrhage or Hematoma
Electronic Medical Records 0.0503 0.0866 0.1916 -0.1513
PACS -2.2799 -0.0079 -0.3930 -0.0645
Clinical/Nursing IT -0.0213 0.0717 -0.0245 -0.0113
Postoperative Pulmonary Embolism or DVT
Electronic Medical Records 0.0870 0.3299 0.9670** 0.5225
PACS 2.3452 0.3784 0.9017 0.8966
Clinical/Nursing IT 0.1933 -0.2254 -0.2466 0.0188

* denotes significance at p=0.10, ** at p=0.05, and ***at p=0.001
Note: Health IT effects are lagged one year

Slide 15

Main Findings

Main Findings

  • EMR investments improve patient safety by reducing infections due to medical care.
  • Others (PACS & Nurse Charting) Health IT Systems are not as effective as EMR.
  • EMR's affect on patient safety grows with time.
  • We find limited evidence wide-spread HIT value.

Slide 16

Contribution and Policy Implications

Contribution & Policy Implications

  • Demonstrates the use of large scale claims data analysis to study health IT impact. More could be done:
    • More recent years.
    • Other populations besides Medicare.
  • Evidence suggest savings will not be quite as big as projected.
  • For new initiatives that are part of health reform, it will be critical for them to show their value using nationally generalizable data, since it is available for analysis and the fiscal stakes have not been higher.

Slide 17

Going Forward

Going Forward

  • Use National Data to track how IT investments are influencing measure outcomes with claims data linked to available clinical data Today (e.g., lab results and Imaging URLs) for appropriate CPT codes.
  • If Phase IV Post-launch drugs and devices can use claims data for monitoring effectiveness of treatments and avoidance of adverse events—why not the federal government with Medicare Parts A, B and D data.
  • Zhan & Miller (2003) set a great precedent for AHRQ to publish the code to measure medical errors. There needs to be far more efforts in this direction to gauge national impact of health IT.

Slide 18

Today's World vs. What $30 billion better build

Today's World vs. What $30 billion better build

Image: Several blocks of SimCity® buildings are shown with captions noting what types of buildings they are: Federal Government, Congress, Main Street, Medical Technology, Big Business, Hospitals, Physicians, Courts, Insurers/Banks, less than 90% income, 91-99% income, 99% income. Yellow lines, representing Today's World, connect Insurers/Banks to Hospital and Physicians; Hospital and Physicians are also connected by a yellow dotted line. Red lines, representing What $30 Billion Better Build connect Insurers/Banks to less than 90% income, Federal Government, Main Street, Medical Technology, and Big Business, and connect Hospitals and Physicians to Medical Technology, and all levels of income.

Slide 19

If Government Really Wants HIT Acceleration - Consider:

If Government Really Wants HIT Acceleration—Consider:

  • What: Federal/State health benefits require providers to pay using a national health card technology platform.
  • Government Why: Want clinical data attached to claim for de-identified comparative effectiveness data pipeline. Bonus—technology platform to mitigate prevent fraud as 'pay for'.
  • Provider Why: I'll get paid in 4 days and under for 90% claims.
  • The Big How:
    • Augment Medicare Administrative Contracts (MACs) for 2011-12 to include card use and require linked clinical data for approximately 100 HCPCS/CPT codes as pilot—more later.
    • Augment TRICARE contracts to do the same.
    • States put contracts out for competitive bid following TRICARE model.
    • FEHBP buts this required specification as well.

Slide 20

Next Steps to Dead End:

Next Steps to Dead End:
Integrated Health Care Demonstration Project R-18

  • Proposed a Trial Integrated Health Card (combine clinical data with claims transmission for new locations:
    • University of Minnesota employees and dependents (>35,000 lives).
    • Minnesota Care (Minnesota's variant of state Medicaid program).
  • Additional support:
    • Metavante—Issues card technology / payment hub (without clinical link) for >30K lives including all of Minnesota's public programs and other state programs—would provide all demonstration technology and consulting support gratis.
  • Comments from Study Panel Reviewers (paraphrased):
    • 1) Claims data is inferior for measuring outcomes and should be not be encouraged as a platform because it is not consistent with an ACO.
    • 2) The researchers are well regarded but the technology (e.g., Medavante—> $10 billion firm) does not exist.
    • 3) Great idea—go for it—Best ebayer ever A+++++++
  • My question—Other than building a company and evaluating it (my current hobby), could this ever be expedited for demonstration funds given the stakes at end.
Current as of December 2010
Internet Citation: Health Information Technology and Patient Outcomes: AHRQ Sponsored Evidence and Next Steps (Text Version). December 2010. Agency for Healthcare Research and Quality, Rockville, MD.


The information on this page is archived and provided for reference purposes only.


AHRQ Advancing Excellence in Health Care