Skip Navigation Archive: U.S. Department of Health and Human Services U.S. Department of Health and Human Services
Archive: Agency for Healthcare Research Quality www.ahrq.gov
Archival print banner

This information is for reference purposes only. It was current when produced and may now be outdated. Archive material is no longer maintained, and some links may not work. Persons with disabilities having difficulty accessing this information should contact us at: https://info.ahrq.gov. Let us know the nature of the problem, the Web address of what you want, and your contact information.

Please go to www.ahrq.gov for current information.

Patient-Reported Outcomes for Adverse Event Monitoring in Clinical Research (Text Version)

Slide presentation from the AHRQ 2010 conference.

On September 28, 2010, Ethan Basch made this presentation at the 2010 Annual Conference. Select to access the PowerPoint® presentation (3 MB).


Slide 1

 Slide 1. AHRQ Annual Conference: Patient-Reported Outcomes for Adverse Event Monitoring in Clinical Research.

AHRQ Annual Conference

Patient-Reported Outcomes for Adverse Event Monitoring in Clinical Research

Ethan Basch, M.D., M.Sc.
Memorial Sloan-Kettering Cancer Center

September 28, 2010

No Financial Disclosures

Slide 2

 Slide 2. Adverse Event Monitoring

Adverse Event Monitoring

Essential activity in Clinical Research:

  • To ensure patient safety.
  • To provide data about drug effects:
    • Trialists, regulators, payors, clinicians, patients

Core activity in routine care:

  • To guide therapy and supportive care

Slide 3

Slide 3. Data Sources Differ By Type of AE.  

Data Sources Differ By Type of AE

Category Example Data Source
Laboratory value Anemia Lab report
Clinical observation/measurement Retinal tear Clinical staff
Symptom Nausea Clinical staff vs. patients

Slide 4

 Slide 4. Clinicians systematically downgrade symptoms compared with patients.

Clinicians systematically downgrade symptoms compared with patients

Image: Six line graphs show Patient-reported vs. Clinician-reported is shown for 6 conditions: fatigue, anorexia, nausea, vomiting, diarrhea, and constipation. For each of condition, patients report more symptoms than clinicians.

Basch: NEJM 2010.

Slide 5

 Slide 5. Patient adverse symptom reports better correlate with functional status than clinician reports.

Patient adverse symptom reports better correlate with functional status than clinician reports

Image: A line chart shows strength of concordances for patient-reported and clinician-reported symptoms. Data is presented in the following table:

Symptom Patient-reported
Strength of Concordance
Clinician-reported
Strength of Concordance
Fatigue 0.36 0.24
Nausea 0.19 0.10
Vomiting 0.13 0.09
Diarrhea 0.14 0.05
Constipation 0.17 0.13
Dyspnea 0.27 0.15
Appetite Loss 0.28 0.22

Basch: JNCI 2009.

Slide 6

 Slide 6. Clinician Adverse Symptom Reporting is Unreliable

Clinician Adverse Symptom Reporting is Unreliable

Symptom ICC 95% CI
Constipation 0.48 0.36; 0.58
Diarrhea 0.58 0.49; 0.66
Dyspnea 0.69 0.62; 0.75
Fatigue 0.50 0.39; 0.59
Nausea 0.52 0.41; 0.60
Neuropathy 0.71 0.65; 0.76
Vomiting 0.46 0.34; 0.56
  • N=393
  • Seen by 1st clinician in office, then 2nd clinician ~15 minutes later

Atkinson: SBM 2010.

Slide 7

 Slide 7. Current Model for Adverse Symptom Reporting in Clinical Trials.

Current Model for Adverse Symptom Reporting in Clinical Trials

Image of a flowchart with the following information:

Step 1: Patient experiences symptoms, then Clinician interviews patient at visit.
Step 2: Clinician interprets symptoms, then Clinician writes in chart.
Step 3: Chart representation of symptom, then Data manager abstracts chart, converts findings to standardized terminology.
Step 4: Data manager interpretation of symptom, then manual data entry.
Last step: the data is entered into the research database.

Slide 8

 Slide 8. Image of The "Patient Experiences Symptom to Research Database"

Image: The "Patient Experiences Symptom to Research Database" is shown. A text box labeled "Patient experiences symptoms" is in the upper-left corner of the slide, and an arrow points down to a data bin labeled "Research database" in the lower right corner. The arrow is captioned "Patient direct reporting of symptoms."

Slide 9

 Slide 9. Image of "Patient Experiences Symptom to Research Database and Clinician"

Image: "Patient Experiences Symptom to Research Database and Clinician" is shown. A text box labeled "Patient experiences symptoms" is in the upper-left corner of the slide, and an arrow points down to a data bin labeled "Research database" in the lower right corner. The arrow is captioned "Patient direct reporting of symptoms." A second, uncaptioned arrow points from "Patient experiences symptom" to a text box labeled "Clinician."

Slide 10

 Slide 10. Available Technologies

Available Technologies

  • Web-based
  • Handheld devices
  • IVRS
  • Paper
  • Text messaging
  • Interviewer
  • Mixed methods/modes catering to patients

Slide 11

 Slide 11. Patient Experiences Symptom to Research Database and Clinician

Image: A text box labeled "Patient experiences symptoms" is in the upper-left corner of the slide, and an arrow points down to a data bin labeled "Research database" in the lower right corner. The arrow is captioned, "Patient direct reporting of symptoms." A second, uncaptioned arrow points from "Patient experiences symptom" to a text box labeled "Clinician." A broken line leads from the "Clinician" box to the "Research Data" bin; this line is captioned, "Assign attribution; initiate expedited reporting."

Slide 12

 Slide 12. Patient Experiences Symptom to Research Database and Clinician

Image: A text box labeled "Patient experiences symptoms" is in the upper-left corner of the slide, and an arrow points down to a data bin labeled "Research database" in the lower right corner. The arrow is captioned, "Patient direct reporting of symptoms." A second, uncaptioned arrow points from "Patient experiences symptom" to a text box labeled "Clinician." There is also a broken-line arrow pointing back from the "Clinician" box to the "Patient experiences symptoms" box. A broken line leads from the "Clinician" box to the "Research Data" bin; this line is captioned, "Assign attribution; initiate expedited reporting."

Slide 13

 Slide 13. Patient Self-Reporting is Already Standard in Closely-Related Areas

Patient Self-Reporting is Already Standard in Closely-Related Areas

  • HRQL and symptom efficacy endpoints in cooperative group trials.
  • Gold standard for symptom endpoints in drug applications and labeling claims submitted to FDA.

Image: To the right of the above text is the publication cover for "Guidance for Industry: Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims". This is an FDA publication from December 2009 from the Center for Drug Evaluation and Research (CDER), Center for Biologics Evaluation and Research (CBER), and the Center for Devices and Radiological Health (CDRDH). There is also a link to: http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM193282.pdfz [Plugin Software Help]

Slide 14

 Slide 14. Criticisms of Patient-Reported AEs

Criticisms of Patient-Reported AEs

  • Not feasible:
    • Patients not willing or able to report.
    • Missing data when patients become ill.
    • Too logistically cumbersome/expensive.
  • Will generate "noise":
    • Patients will broadly endorse symptoms if asked, making it impossible to distinguish AEs between study arms.
    • Will not be helpful in unmasking serious or unexpected AEs.

Slide 15

 Slide 15. Feasibility

Feasibility

  • High rates of adherence in multi-center industry trials for patient-reported symptoms (IVRS).

Image: A bar chart shows mean and median rates of adherence for different patient populations. Data is provided in the table below:

Population Mean Median
Young 79% 83%
Adult 88% 92%
Elderly 91% 96%
Male 87% 93%
Female 87% 92%
<8 Assess 91% 95%
>8 Assess 86% 92%
Fibromyalgia 85% 91%
Osteoarthritis 91% 95%
Post Herpetic Neuralgia 90% 95%

Meacham & Wenzel (Perceptive Informatics/ClinPhone): ISPOR, 2008.

Slide 16

 Slide 16. Feasibility

Feasibility

  • Little attrition over time (Web -based)
    • Including non-Web avid, elderly, end-stage with high symptom burdens

Image: A bar chart titled "Proportion of Patients Completing Online Questionnaire at a Given Clinic Visit (%)" is shown. The chart is organized by visit numbers 1-24. It ranges from a high of 100% (Visit 1) down to 65% (Visit 15).

Basch: JCO 2005; 2007.
Velikova: JCO 2002.
Farnell: Eur J Cancer 2010.

Slide 17

 Slide 17. Pros Can Distinuish between Study Arms and Identify Serious AEs

PROs Can Distinguish between Study Arms and Identify Serious AEs

  • NCCTG 9741: Phase III trial comparing three chemotherapy regimens for metastatic colorectal cancer.
  • Closed after 841/1,125 patients enrolled due to unexpected excess of early deaths in Arm 1 ("IFL")
    • Associated with "GI syndrome" including severe diarrhea.
  • Diarrhea reporting:
    • Clinicians reported toxicities at each cycle (diarrhea required).
    • Patients reported diarrhea via in HRQL (SDS) every other cycle.

Rothenberg: JCO, 2001.

Slide 18

 Slide 18. Clinician-Reported Diarrhea

Clinician-Reported Diarrhea

Image: A line graph shows event-free probability by the Arm cycle. All three Arms begin at 100% event-free probability. Arm 1 drops below 80% in the first cycle and continues to drop to ~60% by cycle 14. Arm 2 (HR = 0.29 vs. Arm 1, P-value <0.001) drops only slightly from cycle to cycle and levels off at ~90% by cycle 7, where it remains until cycle 20 at the end of the chart. Arm 3 (HR = 0.77 vs. Arm 1, P-value <0.05) drops below 80% in the first cycle and continues to drop to 65% by cycle 10; it then remains level until cycle 20.

Dueck: Unpublished Data, 2010.

Slide 19

Slide 18. Clinician Reported Diarrhea  

Patient-Reported Diarrhea

Image: A line graphs shows event-free probability by the Arm cycle. All three Arms begin at 100% event-free probability. Arm 1 drops slightly after the first cycle, then drops sharply to ~65% by cycle 2; thereafter, it drops slightly in each subsequent cycle to end at ~30% in cycle 10. Arm 2 (HR = 0.36 vs. Arm 1, P-value <0.001) drops to ~90% by cycle 3, then below 80% by cycle 4; thereafter, it drops slightly in each subsequent cycle to end just above 50% in cycle 20. Arm 3 (HR = 0.68 vs. Arm 1, P-value <0.003) drops to ~90% in cycle 2, then drops sharply to ~70% by cycle 3; thereafter, it drops slightly in each subsequent cycle to end at ~ 30% in cycle 20.

Dueck: Unpublished Data, 2010.

Slide 20

Slide 20. Patient vs. Clinician Diarrhea in Arm 1 (IFL)  

Patient vs. Clinician Diarrhea in Arm 1 (IFL)

Image: A line chart shows event-free probability for Arm 1 (IFL). Both the Clinician- and Patient-reported lines begin at 100% event-free probability. The Clinician-reported line drops to 80% in the first cycle and continues to drop to ~60% by cycle 14. The Patient-reported line drops slightly after the first cycle, then drops sharply to ~65% by cycle 2; thereafter, it drops slightly in each subsequent cycle to end at ~30% in cycle 10.

Dueck: Unpublished Data, 2010.

Slide 21

 Slide 21. Adverse Symptoms are Common

Adverse Symptoms Are Common

  • Many adverse reactions in drug labels are symptoms.
Indication # of U.S. Approved Drug Labels Average # of AEs per Label Total # of Unique AEs across Labels Proportion of AEs Which are Symptoms
Breast Cancer 32 78 616 36% (223/616)
Asthma 35 54 368 49% (180/368)
GERD 18 115 472 45% (213/472)
Hyperlipidemia 28 82 365 43% (158/365)
Osteoarthritis 39 94 684 41% (278/684)

Slide 22

 Slide 22. Docetaxel Drug Label

Docetaxel Drug Label

Image: A Docetaxel Drug Label with probabilities of adverse events circled in red is shown.

Slide 23

 Slide 23. NCI Contract HHSN261200800043C

NCI Contract HHSN261200800043C

Development of the Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE)

  • Initiated October 2008

Slide 24

 Slide 24. Mission

Mission

Develop a system for patient electronic self-reporting of adverse symptoms in cancer trials, which is widely accepted and used; generates useful data for investigators, regulators, clinicians and patients; and is compatible with existing adverse event reporting software systems.

Slide 25

 Slide 25. PRO-CTCAE Network

PRO-CTCAE Network

Image: A chart depicting the PRO-CTCAE Network is shown. The MSKCC Coordinating Center works with the following components:

NCCCP:
Christiana
Hartford
OLOL
Spartanburg
St. Joseph-Orange

NCI:
DCCPS
DCP
DCTD
CTIIT

Network:
Dana-Ferber
MD Anderson
Mayo
Duke
Penn

Tech:
SemanticBits
Perceptive

Advisors:
Cooperative Groups
FDA
Industry
Patient Advocates

Slide 26

 Slide 26. Item Development

Item Development

  • Evaluated the standard lexicon for adverse event reporting in oncology (CTCAE):
    • Currently reported by clinicians.
  • Of 790 CTCAE items, 81 are amenable to patient self-reporting ("symptoms"):
    • To create patient versions of these items, generic question structures were developed based on existing questionnaires
      • Removed medical jargon.
      • Attention to cultural literacy

Slide 27

 Slide 27. Example: Mucositis

Example: Mucositis

CTCAE v4 Term Grade 1 Grade 2 Grade 3 Grade 4
Mucositis oral Asymptomatic or mild symptoms; intervention not indicated Moderate pain; not interfering with oral intake; modified diet indicated Severe pain; interfering with oral intake Life-threatening consequences; urgent intervention indicated
Two PRO-CTCAE v1 Items Responses
What was the severity of your Mouth or Throat Sores at their worst? None
Mild
Moderate
Severe
Very Severe
How much did Mouth or Throat Sores interfere with your usual activities? Not at all
A little bit
Somewhat
Quite a bit
Very much

Slide 28

 Slide 28. Item Refinement

Item Refinement

  • Multicenter "cognitive interviewing" study to assure comprehension across diverse patient populations.
  • National "validation" study underway to evaluate measurement properties of items.

Hay: ASCO 2010.
Dueck: ASCO 2010.

Slide 29

Slide 29. Software Platform  

Software Platform

Image: A chart depicting the Software Platform is shown. A patient, clinician, or administrator can use a PC, Tablet PC, or PDA via various interfaces that allow them to perform the following specific tasks:

Patient Interface:

  • Fill out questionnaires.

Clinician Interface:

  • Build standard questionnaires.
  • Manage patients.
  • Configure rules based notifications and alerts.
  • Schedule questionnaires.

Report and Analytics Interface (available to clinicians):

  • Generate patient-level reports.
  • Generate study-level reports.

Administrator Interface:

  • Administer security policies.
  • Provision clinical staff.
  • Configure system.

The software is built on:

  • Web 2.0
  • Rules Engine
  • Spring Framework
  • Hibernate
  • Oracle
  • PostgreSQL

Slide 30

 Slide 30. Image: A screen shot of the Pro-CTCAE Patient Symptom Reporter Web site is shown. The Form Builder page is featured.

Image: A screen shot of the Pro-CTCAE Patient Symptom Reporter Web site is shown. The Form Builder page is featured.

Slide 31

 Slide 31. Image: A screen shot of the Pro-CTCAE Patient Symptom Reporter Web site is shown. The Schedule Form page is featured.

Image: A screen shot of the Pro-CTCAE Patient Symptom Reporter Web site is shown. The Schedule Form page is featured.

Slide 32

Slide 32. Image: A screen shot of the Pro-CTCAE Patient Symptom Reporter Web site is shown. A questionnaire page is featured.  

Image: A screen shot of the Pro-CTCAE Patient Symptom Reporter Web site is shown. A questionnaire page is featured.

Slide 33

 Image: A screen shot of the Pro-CTCAE Patient Symptom Reporter Web site is shown. The Symptom summary page is featured.

Image: A screen shot of the Pro-CTCAE Patient Symptom Reporter Web site is shown. The Symptom summary page is featured.

Slide 34

Slide 34. Image: A screen shot of the Pro-CTCAE Patient Symptom Reporter Web site is shown. An auto-generated E-mail is featured  

Image: A screen shot of the Pro-CTCAE Patient Symptom Reporter Web site is shown. An auto-generated E-mail is featured.

Slide 35

Slide 35. Survey  

Survey

729 Stakeholders in Cooperative Groups

Question Agree Neutral Disagree
Systems to collect PROs in trials should be developed 89% 5% 6%
In trials, adverse events should be reported by patients 88% 8% 4%
 
Potential Barriers Agree Neutral Disagree
Lack of computers 69% 15% 16%
Limited personnel 57% 18% 25%
 
Solutions to Overcome Barriers Agree Neutral Disagree
Funding (for personnel, dedicated space, training) 79% 13% 8%
Computers 72% 21% 7%

Bruner et al: ISOQOL, 2010.

Slide 36

Conclusions

Slide 36. Conclusions

  • Electronic patient-reporting of adverse symptoms in clinical trials is feasible and clinically valuable:
    • Improve quality and efficiency of safety data collection.
    • Enhance understanding of patient experience with treatment.
    • Alert investigators and clinicians to issues meriting attention
  • Appropriate for multiple contexts:
    • Preapproval clinical trials.
    • Postmarket surveillance.
    • Comparative effectiveness research.
    • Clinical practice.
  • Ongoing efforts to operationalize and pilot systems.
Current as of December 2010
Internet Citation: Patient-Reported Outcomes for Adverse Event Monitoring in Clinical Research (Text Version). December 2010. Agency for Healthcare Research and Quality, Rockville, MD. http://archive.ahrq.gov/news/events/conference/2010/basch/index.html

 

The information on this page is archived and provided for reference purposes only.

 

AHRQ Advancing Excellence in Health Care