Skip Navigation Archive: U.S. Department of Health and Human Services U.S. Department of Health and Human Services
Archive: Agency for Healthcare Research Quality
Archival print banner

This information is for reference purposes only. It was current when produced and may now be outdated. Archive material is no longer maintained, and some links may not work. Persons with disabilities having difficulty accessing this information should contact us at: Let us know the nature of the problem, the Web address of what you want, and your contact information.

Please go to for current information.

Experience in improving healthcare decision-making with health IT: integrating theory, research, and practice (Text Version)

Slide presentation from the AHRQ 2009 conference.

On September 14, 2009, Matthew Samore, MD made this presentation at the 2009 Annual Conference. Select to access the PowerPoint® presentation (537 KB).

Slide 1

Experience in improving healthcare decision-making with health IT: integrating theory, research, and practice

Matthew Samore, MD
VA Salt Lake City Health Care System
Professor of Internal Medicine
Adjunct Professor of Biomedical Informatics
University of Utah


Slide 2


  • Salt Lake VA Informatics, Decision Enhancement, ,and Surveillance (IDEAS) Center Selected Investigators and Collaborators
  • Jonathan Nebeker, MD
  • Charlene Weir, PhD
  • Frank Drews, PhD
  • Michael Rubin, MD, PhD
  • Kim Bateman, MD
  • Brian Sauer, PhD
  • Lucy Savitz, PhD
  • Tom Greene, PhD
  • R. Scott Evans, PhD
  • Randall Rupper, MD, MPH

Partners: University of Utah, VA Salt Lake City, Healthinsight, CaduRx, Intermountain Healthcare
VA HSR&D REA 08-264
AHRQ R01 HS15413
AHRQ 1R18HS017308-01


Slide 3

Thesis of this talk

Theory and models provide scientific underpinnings for generalization
- Which supports comparative effectiveness research
For health services research and epidemiology:
- Use of models understood
For clinical decision support:
- Not so much
Health information technology ≠ informatics
- Sub-disciplines such as cognitive informatics crucial


Slide 4

More succinctly expressed:

"The difference between theory and practice is that in theory there is no difference but in practice there is"


Slide 5

Statement of the problem:

"A disproportionate amount of literature on the benefits [of health information technology] that have been realized comes from a small set of early-adopter institutions that implemented internally developed health information technology systems.."

Chaudhry et. al. Ann Intern Med. 2006;144:742-752


Slide 6

Addressing generalizability

  • In what contexts will effects generalize?
  • What accounts for variability in results?
  • Why are impacts lower in magnitude or narrower in scope in larger trials compared to single institution studies?
  • How to incorporate information about implementation, adoption, formative evaluation?


Slide 7

Relevance to comparative effectiveness research (CER)

  • CER priorities
    • Directly focused on health information technology
    • Compare the effectiveness of alternative redesign strategies-using decision support capabilities, electronic health records, personal health records
  • Indirectly tied to health information technology
  • Compare the effectiveness of various strategies
    • To control MRSA
    • To control healthcare associated infection
    • To enhance patients' adherence to medication regimens


Slide 8

Addressing CER challenges

Need to explicitly formulate causal question

  • Determining identifiability
  • Defining level of inference
  • Validating methods to reduce bias


Slide 9

Conceptual frameworks (THEORY)

Natural & engineered systems
- Co-evolution
Cognitive processing
- Information overload �" fit-to-workflow
Cyclical models of control
- Feedback and feed-forward


Slide 10

System co-evolution

Fundamental theorem in informatics
- C. Friedman J Am Med Inform Assoc. 2009;16:169-170
Proposed modification:
- Computers plus humans create a distinct socio-technical system
  -Characteristics are not equivalent to other industries


Slide 11


Level of inference needed to assess causal effect of health information technology:
- Socio-technical system
Potential benefits of simulation


Slide 12

Cognitive processing

Motivation, mental models, tasks, goals
- Influenced by social context
Lack of fit-to-workflow experienced as:
- Information overload
- Interruptions


Slide 13


Cognitive informatics methods
- Task analysis
- Direct observation
- Match implementation strategy to task complexity


Slide 14

Second law of thermodynamics as applied to cognition:

Humans seek states of reduced cognitive effort
- Workarounds
As cognitive load increases, automatic processing systems kick-in


Slide 15

For those who believe that there is a Simpson's quote for every situation

"In this house, we obey the laws of thermodynamics!

Homer Simpson's response when his daughter builds a perpetual motion machine in which energy increases with time


Slide 16

Contextual Control Model

  • Feed-back systems not sufficient
    • Need to anticipate and predict
    • Pure feedback systems subject to loss of stability
    • Time horizon is long in strategic control modes
  • Relevance
    • Link between decision support and surveillance
    • Surveillance contributes feedback and feed-forward capabilities


Slide 17

Feedback & feed-forward decision support


Slide 18

Illustrative experience with decision support for antimicrobial prescribing

  • Two different technologies studied
  • Clinical task:
    • Management of patient with acute respiratory infection in outpatient setting
      • Whether or not to prescribe an antibiotic
      • Choosing the antibiotic
      • Diagnostic label
      • Impact of perceived or actual patient demand


Slide 19

Application of theory to practice
implementation of electronic health records in rural settings

  • Socio-technical system
    • Hook was electronic prescribing
  • Stepwise approach to adoption
  • Accommodating variation
    • Readiness to change
  • Social context and clinic culture
  • Encouraging play
  • Avoiding information overload


Slide 20

Community intervention plus clinical decision support system

  • Standalone algorithms on handheld computers
  • Community randomized trial

Samore MH et. al. JAMA. 2005 Nov 9;294(18):2305-14.
Effect on prescribing any antibiotic


Slide 21

Clinical decision support system integrated with computerized clinic order entry

Effect on macrolide prescriptions

  • Algorithm usually triggered by ordering antibiotic
  • Clinic randomized trial


Slide 22


  • Deciding whether to prescribe an antibiotic and choosing the drug involve different cognitive processes
  • Given that decision to prescribe an antibiotic is made
    • Possible to embed correct choice in workflow
  • Feed-forward decision support needed to impact the "is this a situation that warrants an antibiotic" decision
  • Relevant to drug-drug interaction alerting


Slide 23

Recommendations and conclusions

Models fundamental to translation of research into practice
Incorporation of theory and models into comparative effectiveness research
Role of simulation

Current as of December 2009
Internet Citation: Experience in improving healthcare decision-making with health IT: integrating theory, research, and practice (Text Version). December 2009. Agency for Healthcare Research and Quality, Rockville, MD.


The information on this page is archived and provided for reference purposes only.


AHRQ Advancing Excellence in Health Care