Skip Navigation Archive: U.S. Department of Health and Human Services U.S. Department of Health and Human Services
Archive: Agency for Healthcare Research Quality
Archival print banner

This information is for reference purposes only. It was current when produced and may now be outdated. Archive material is no longer maintained, and some links may not work. Persons with disabilities having difficulty accessing this information should contact us at: Let us know the nature of the problem, the Web address of what you want, and your contact information.

Please go to for current information.

Mortality Measurement: Slide Presentation (Text Version)

Slide Presentation for Mortality Measurement Meeting

Presentations from a November 2008 meeting to discuss issues related to mortality measures.

Slide Presentation for Mortality Measurement Meeting

On November 3, 2008, Gabriel Escobar, M.D., and Marta Render, M.D., made this presentation at the 2008 Annual Conference. Select to access the PowerPoint® presentation (94 KB).

Slide presentations can be accessed using a Free PowerPoint® Viewer).

Slide 1

Mortality Measurement in the Electronic Medical Record (EMR) Era: What Real Time Lab and Clinical Data Can Contribute to Precision and Prediction

Gabriel J. Escobar, MD
Marta Render, MD
John D. Greene, MA
Marla N. Gardner, BA
Patricia Kipnis, PhD

Slide 2

Topics to be Covered

  • Value of readily available enhancements:
    • Present-on-Admission (POA) coding
    • Laboratory data
    • Other physiologic data
  • Work in progress: use of bed history data combined with laboratory data

Slide 3

Why should we settle for the lowest common denominator?

  • Billing data are driven by need for reimbursement
  • High propensity to being "gamed"
  • "Coding creep" occurs
  • Many diagnoses strongly associated with outcome can be present on admission, e.g., stroke, deep vein thrombosis (DVT), pressure ulcers

Slide 4

Glance et al.: Impact of Present on Admission Coding on Performance Ranking

  • POA coding modifier showed that diagnoses associated with complications were present on admission only 10—22% of the time.
  • Absence of POA coding led to 33—40% of low performing hospitals not being detected for these conditions.
    • Coronary artery bypass grafting (CABG)
    • Coronary angioplasty
    • Hip replacement
    • Acute myocardial infarction (AMI)
  • "Treating complications as pre-existing conditions gives poor-performing hospitals 'credit' for their complications and may cause some hospitals that are delivering low-quality care to be misclassified as average- or high-performing hospitals."

Slide 5

Value of Incorporating Physiologic Data

  • Increasingly available, particularly in hospital chains
  • Much less expensive than manual chart abstraction
  • Have tremendous face validity with clinicians
  • Relatively easy to combine with other data
  • Can be used either for disease-specific models (e.g., Fine Patient Safety Indicators (PSI) for community acquired pneumonia) or for global risk adjustment (e.g., Department of Veterans' Affairs (VA) or Kaiser Permanente risk adjustment methodologies)

Slide 6

Relative Contribution of Physiologic Data to Overall Model Performance

  • Operational use of laboratory data first occurred in the intensive care unit (ICU).
  • Render et al.—29,377 consecutive first ICU admissions in 17 VA hospitals:
    • Laboratory data accounted for 74% of model predictive ability
    • Diagnosis accounted for 13%
  • Zimmerman et al.—110,558 ICU admissions in 45 U.S. hospitals:
    • Laboratory data accounted for 65% of model predictive ability
    • Diagnosis accounted for 16%

Slide 7

Pine et al.: Use of Laboratory Data for Non-ICU Populations

  • Quantified effect of adding POA coding, laboratory data, and vital signs data for 5 conditions and 3 procedures.
  • Not restricted to ICU; N ranged from 5309 for abdominal aortic aneurysms (AAA) to 200,506 for congestive heart failure (CHF).
  • Average effect of adding predictors, as evidenced by change in c statistic:
No risk adjustment0.50
Administrative model0.79
POA model0.84
POA + labs0.86
POA + labs + V S S0.88

Slide 8

Tabak et al.: Definitive Quantification of Value of Laboratory Data

  • Evaluated 6 disease-specific mortality predictive models for pay-for-performance (ischemic and hemorrhagic stroke, pneumonia, CHF, and sepsis).
  • 194,903 admissions in 2000-2003 across 71 hospitals that imported laboratory data.
  • Quantified relative contribution with omega statistic.
  • Laboratory data were between 2 and 67 times more important in predicting mortality than ICD-9 variables.
  • Only models where laboratory data were less important were those for stroke, where altered mental status recordings were more important.

Slide 9

Research in Progress: Combining Laboratory Data with Another EMR Marker, Bed History

  • 207,922 hospitalizations at 19 Kaiser Permanente hospitals, 11/1/06-1/31/08
  • All severity scored using laboratory acute physiology score (LAPS) and pre-admission comorbidity point score (COPS)
  • Employs time stamps for patient arrival at different units (ward, OR/PAR, TCU, ICU)
  • Examines mortality of intra-hospital transfers combined with laboratory testing patterns

Slide 10

Categorization of Intra-hospital Transfers to a Higher Level of Care

  • Post-surgical (OR/PAR to TCU, OR/PAR to ICU)
  • Unplanned (ward to TCU, ward to ICU, TCU to ICU)
  • Laboratory testing patterns (blood gases, lactate, troponin I, blood culture)
    • Not tested during 16 hour (12-4) time window surrounding transfer
    • Tested during time window but not before
    • Tested before and during time window

Slide 11

Testing patterns

  • If index test obtained in 16 hour window (12 hrs before to 4 hours after transfer to higher level of care) patient is considered to have been tested, otherwise patient is considered "not tested."
  • If another test is located within 24-48 hours (depending on test type) preceding index test, the patient is considered "previously tested."
  • If another test is not located within 24-48 hours preceding index test, the patient is considered "newly tested."

Slide 12

Outcomes Based on First Hospital Unit

LAPS (median)1802028
COPS (median)79538981
Mortality (mean,p)3.9%0.8%4.7%8.9%
Mortality (mean,a)3.2%0.5%4.4%12.5%

Slide 13

Outcomes Based on Intra-hospital Transfer Type—I

GroupN (%)Death RateOEMR
Never in TCU or ICU155,298(75%)1.6%0.57
Direct admit to ICU (all)15,951(7.7%)12.5%1.41
Experienced unplanned transfer786(0.4%)24.1%2.03
Direct admit to TCU (all)20,416(9.8%)4.4%0.94
Experienced unplanned transfer1,388(0.7%)24.1%2.92

Slide 14

Outcomes Based on Intra-hospital Transfer Type—II

GroupN (%)Death RateOEMR
Post-surgical transfers to TCU or ICU    
Not Tested6,458(3.1%)1.7%0.79
Newly Tested1,385(0.7%)6.4%1.78
Previously Tested2,452(1.2%)6.4%1.84

Slide 15

Outcomes Based on Intra-hospital Transfer Type—III

GroupN (%)Death RateOEMR
Unplanned transfer to TCU or ICU    
Not Tested4,311(2.1%)10.2%1.84
Newly Tested2,439(1.2%)25.4%3.35
Previously Tested2,621(1.3%)26.6%3.39

Slide 16

OEMR For Unplanned Transfers in 17 Kaiser Permanente Hospitals

Chart with facility on the x axis and observed to expected mortality ratio on the y axis. The ratios range from 2 to approximately 3.8.

Slide 17

Elapsed Hospital Length of Stay (ELOS) at Time of Death

GroupELOS (days) @ time of death (median, mean plus or minus SD)
Ward Patients3.8, 6.0 plus or minus 8.9
TCU/ICU Direct Admits4.7, 10.1 plus or minus 19.6
Post-Surgical Transfers12.9, 21.8 plus or minus 32.2
Unplanned Transfers10.0, 17.9 plus or minus 26.0

Slide 18

Next EMR Challenge: Identification of Do-Not-Resuscitate (DNR)/Comfort Care Patients

Holloway and Quill:

  • Mortality is a good quality measure for individuals with acute illness who are not supposed to die but is a poor quality measure for the majority of patients with multiple chronic diseases who are near the end of their life.
  • "[T]aken alone, short-term mortality measures essentially treat death as a medical failure and reinforce avoiding death at all costs."
  • Careful use of EMRs could permit excluding these patients from analyses (or separating them for different types of analyses).

Slide 19


  • Evidence base for use of physiologic data in risk adjustment is overwhelming.
  • Two entities, the VA system and Kaiser Permanente, are already employing physiologic-based risk adjustment operationally.
  • Other uses of physiologic data (e.g., multivariable template matching for targeted case-control studies; VA's use of change in serum creatinine) make the use of these data even more compelling
  • The important question is not "Should one employ automated physiology-based risk-adjustment?" but, rather, "Is it possible to convince different institutions to standardize data definitions so as to permit larger collaborative studies?"

Slide 20


  • Entities such as IHI and AHRQ should be creating incentives for use of these data, rather than simply reinforcing the use of administrative data
  • Not all hospitals have the capability to employ these data, but for those that do, IHI/AHRQ should encourage creation of networks that collaborate to employ data from EMRs
  • Future research using EMRs should emphasize capture of D N R/comfort care orders

Slide 21

Veterans Administration Risk Adjusted Mortality in the ICU

  • Risk model
    • Diagnosis, comorbid disease, source of admission, worst of 11 lab values (sodium, glucose, BUN, Creatinine, albumin, Bilirubin, white blood count, hematocrit, pH/PaCO2, PaO2
    • Separate models predict death at 30 days and at hospital discharge
  • Advantage in addition of physiologic data
    • Face validity/improved discrimination calibration to fairly portray risk and outcomes
  • Unexpected advantage of using lab values in model
    • Ability to create metrics related to labs:
      • Mean hyperglycemia.
      • Hypoglycemic rate/ patient days on hypoglycemic agent.
      • Rate of acute kidney injury (AKI).
    • Ability to link relationships:
      • Troponin with mortality in medical patients and/or beta blocker use
      • CHF readmission rates with ACE AKI

The VA Inpatient Evaluation Center

Aggregate VA ICU Standardized Mortality Rate

Chart showing year on the x axis and SMR on the y axis. Years range from 2002 to 2008. SMR ranges from 0 to 1.4. Values are shown for SMR 30 and SMR Hosp, as follows:

SMR 30: 2002, 1.099; 2003, 1.063; 2004, 1.065; 2005, 1.008; 2006, 0.996; 2007, 0.997; 2008, 0.940
SMR Hosp: 2002, 1.194; 2003, 1.152; 2004, 1.144; 2005, 1.059; 2006, 1.012; 2007, 0.977; 2008, 0.923

Slide 22

Mortality data and VA ICUs

  • Multiple mortality measures tell you different things:
    • Unadjusted mortality at hospital discharge or at 30-days for ICU and acute care patients (Are the admission criteria similar; access).
    • Risk adjusted mortality at hospital discharge or at 30-days for ICU and Acute care patients (The difference may address issues of alternative access for placement for chronic acute illness/ futile care).
    • Unadjusted mortality of patients transferred from the ward to the ICU (ability to detect and rescue deteriorating patients).
    • Unadjusted mortality graphed against SMR may tell you about risk of underperforming hospital (vulnerability of the healthcare organization and patients).

Slide 23

Use of the VA ICU mortality measures

  • Reported every quarter; web based; regional access with national benchmarks stratified by level and type of ICU.
  • "Boots on the grounds," Targeted site visits, identification of unmeasured variables, Review of low predicted mortality patients who die, use of evidenced based practices, recommendations, follow-up in 6 months.
  • Case mix index (ICU predicted mortality/all ICU predicted mortality).
    • Adjusted bed turns, Track severity at ICUs with more limited services
  • Targeting groups of patients for ICU LOS reduction (<2.5% predicted mortality)
  • Evaluate effectiveness of initiatives:
    • ACS, SCIP

Slide 24

The association of hyperglycemia with mortality varies by diagnosis

Three charts show odds ratio for mortality for three conditions by mean glucose. Odds ratio is on the x axis and mean glucose (milligrams per deciliter) is on the y axis. For unstable angina, the approximate odds ratios are: mean glucose 111 to 145, 1.7; mean glucose 146 to 199, 2.3; mean glucose 200 to 300, 3; mean glucose greater than 300, 6. Number of deaths is 23, 274. For pneumonia, the approximate odds are: mean glucose 111 to 145, 1.2; mean glucose 146 to 199, 1.9; mean glucose 200 to 300, 1.8; mean glucose greater than 300, 1.8. Number of deaths is 8,449. For sepsis, the approximate odds are: mean glucose 111 to 145, 1.2; mean glucose 146 to 199, 1.5; mean glucose 200 to 300, 2.3; mean glucose greater than 300, 3.4. Number of deaths is 4,179.

Slide 25

Even mild renal injury (Creatinine greater than 0.3 milligrams per deciliter) increases mortality risk and LOS

Disease CategoriesMortality Rate
N (%)
Odds Ratio
(95% CI)
Mean (SD)
Mean (SD)
Overall (n, 310,323)31,912 (10.3)   
No AKI (n, 244,550)15,106 (6.2)Reference4.5 (3.6)- 0.44 (3.3)
Stage I (n, 52,773)10,212 (19.4)2.22 (2.15-2.29)7.1 (6.2)1.4 (5.7)
Stage II (n, 7,744)3,359 (43.4)6.09 (5.74-6.46)10.6 (8.7)5.2 (8.9)
Stage III (n, 3,271)2,052 (62.7)12.51 (11.5-13.7)13.9 (9.9)10.3 (10.5)
Stage III-D (n, 1,985)1,183 (59.6)8.08 (7.24-9.03)16.2 (10.5)11.2 (10.9)

(CI—confidence limits; AKI—acute kidney injury; OMELOS—observed minus expected length of stay; SD—standard deviation)

Slide 26

Articles Cited:

Zimmerman JE, Kramer AA, McNair DS, et al. Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today's critically ill patients. Critical Care Medicine 2006;34:1297-1310.

Render ML, Kim HM, Welsh DE, et al. Automated intensive care unit risk adjustment: results from a National Veterans Affairs study. Critical Care Medicine 2003;31:1638-46.

Glance LG, Dick AW, Osler TM, et al. Accuracy of hospital report cards based on administrative data. Health Services Research 2006;41:1413-37.

Holloway RG, Quill TE. Mortality as a measure of quality. Implications for palliative and end-of-life care. JAMA 2007; 298:802-4.

Tabak Y P, Johannes R S, Silber J H. Using automated clinical data for risk adjustment. Development and validation of six disease-specific mortality predictive models for pay-for performance. Medical Care 2007; 45:789-805.

Pine M, Jordan H S, Elixhauser A, et al. Enhancement of claims data to improve risk adjustment of hospital mortality. JAMA 2007;297:71-6.

Escobar G J, Greene J D, Scheirer P, Gardner M, Draper D, Kipnis P. Risk Adjusting Hospital Inpatient Mortality Using Automated Inpatient, Outpatient, and Laboratory Databases. Medical Care 2008; 46(3):232-9.

Return to Contents
Proceed to Next Section

Page last reviewed March 2009
Internet Citation: Mortality Measurement: Slide Presentation (Text Version): Slide Presentation for Mortality Measurement Meeting. March 2009. Agency for Healthcare Research and Quality, Rockville, MD.


The information on this page is archived and provided for reference purposes only.


AHRQ Advancing Excellence in Health Care