Skip Navigation Archive: U.S. Department of Health and Human Services U.S. Department of Health and Human Services
Archive: Agency for Healthcare Research Quality www.ahrq.gov
Archival print banner

This information is for reference purposes only. It was current when produced and may now be outdated. Archive material is no longer maintained, and some links may not work. Persons with disabilities having difficulty accessing this information should contact us at: https://info.ahrq.gov. Let us know the nature of the problem, the Web address of what you want, and your contact information.

Please go to www.ahrq.gov for current information.

Development and Use of Ambulatory Adverse Event Trigger Tools (Text Version)

Slide presentation from the AHRQ 2010 conference.

On September 29, 2010, Jonathan R. Nebeker made this presentation at the 2010 Annual Conference. Select to access the PowerPoint® presentation (595 KB). 


Slide 1

Development and Use of Ambulatory Adverse Event Trigger Tools

Development and Use of Ambulatory Adverse Event Trigger Tools

Jonathan R. Nebeker

AHRQ Conference
Sept. 29, 2010

Boston University School of Public Health, Boston, MA,
VA Center for Organization, Leadership and Management Research (COLMR), Boston MA
akrosen@bu.edu

Slide 2

Acknowledgements

Acknowledgements

Sponsored by AHRQ Contract No. HHSA290200600012, Task Order Officer: Amy Helwig, MD

  • PI Amy Rosen, PhD
  • Co-PI Jonathan Nebeker, MD, MS
  • Co-Investigators:
    • Stephan Gaehde, MD
    • Haytham Kaafarani, MD, MPH
    • Brenna Long, MA
    • Hillary Mull, MPP
    • Brian Nordberg, BS
    • Steve Pickard, MS
    • Peter Rivard, PhD
    • Lucy Savitz, PhD, MBA
    • Chris Shanahan, MD, MPH
    • Stephanie Shimada, PhD

Slide 3

What's new?

What's New?

  • ADE Surveillance rules:
    • Complex to maximize PPV and Sensitivity.
    • Action-Oriented to make a difference for patient.
  • Use input from system-based focus groups and experts with practical experience.

Slide 4

Study Flow

Study Flow

Image: A flowchart depicts the following process:

Structured design → Delphi Panel → Focus Group → Assessment

Slide 5

Epidemiological Design

Epidemiological Design

Image: A sample form shows a drug-event trigger.

Mull H & Nebeker JR, Informatics Tools for the Development of Action-Oriented Adverse Drug Event Triggers, 2008. AMIA Annual Symposium Proceedings Nov 6:505-9. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2655939/

Slide 6

Example of Complex Trigger

Example of Complex Trigger

  • [Dose change in GFR reducer or renal toxin and subsequent creatinine > 50% from average baseline creatinine value or > 33% from last post dose change creatinine value] AND (No new trimethoprim within 1 day prior to 7 days after creatinine lab value firing) AND (GFR Reducer or renal toxin used within 90 days prior to lab value).
  • Suppress If: Renal toxin or GFR reducer dose reduced 0-7 days after lab) OR ( new creatinine lab result within 1-6days of firing lab value).

Slide 7

Results: Focus Group/Interviews

Results: Focus Group/Interviews

Image: A bar chart displays the following information:

  Likely Unclear Unlikely
Warfarin 92 8 2
Creatinine 40 52 2
Hyperkalemia 25 60 15
Hypokalemia 25 60 15
Platelets 20 65 15
WBC 20 65 15
Delirium 5 40 55

Slide 8

Delphi Highlights

Delphi Highlights

  • 50% national experts, 50% local clinical leaders.
  • Got perspectives that represented emphasis of research or clinical experience.
    • ED doc liked delirium trigger.
  • Various, contradictory input on thresholds.
  • Helpful modifications.
  • Interesting insights on what would be useful.

Slide 9

Analysis

Analysis

  • Sample:
    • Requested sets of 2,000 pts based on exposure to target meds.
    • Got 1,059 to 9,339 → IRB amendment.
    • Restricted to patients with 2 notes in 1 year.
  • Analysis:
    • 1° PPV = number of patients with AE/number of cases triggered.
    • 2° Sensitivity, if prevalence >5% (for 25% CI width) (highest was 4%).

Slide 10

Ignore the CI behind the curtain

Ignore the CI behind the curtain

Image: A line graph shows sensitivity and positive predictive value for trigger systems.

Nebeker JR, Stoddard GJ & Rosen AK, Considering Sensitivity and Positive Predictive Value in Comparing the Performance of Triggers Systems for Iatrogenic Adverse Events, Proceedings of the Trigger and TIDS Expert Meeting, Agency for Healthcare Research and Quality, June 2008. Bethesda, MD. http://www.ahrq.gov/news/events/other/triggers/triggers2.html

Slide 11

End Points

End Points

  • Harm:
    • PSO Harm scale
    • WHO Causality criteria (certain or probable)
    • Explicit criteria
    • Lab values without symptoms are not harm
  • Change in care (useful):
    • e.g., prevented additional lab draw

Slide 12

Trigger Performance

Trigger Performance

Trigger Reviewed/ Flagged Harm PPV (CI) Change PPV (CI) Either PPV (CI)
Creatinine 40/57 0 60%
(43-75%)
60%
(43-75%)
WBC 0 - - -
Platelets 17/17 6%
(0-29%)
53%
(28-77%)
59%
(33-82%)
Warfarin 96/677 13%
(7-21%)
14%
(7-22%)
19%
(12-28%)
Hypokalemia 85/90 17%
(9-26%)
42%
(31-54%)
58%
(46-68%)
Hyperkalemia 50/64 8%
(2-19%)
28%
(16-43%)
36%
(23-51%)
Delirium 68/359 10%
(4-20%)
10%
(4-20%)
19%
(11-31%)

Slide 13

Summary

Summary

  • Harm is in the eye of the beholder: less or more?
    • Doomed to benchmarking apples with oranges?
    • Explicit criteria.
  • Potential to affect practice.
  • Are complex trigger better?
Current as of December 2010
Internet Citation: Development and Use of Ambulatory Adverse Event Trigger Tools (Text Version). December 2010. Agency for Healthcare Research and Quality, Rockville, MD. http://archive.ahrq.gov/news/events/conference/2010/nebeker/index.html

 

The information on this page is archived and provided for reference purposes only.

 

AHRQ Advancing Excellence in Health Care