New and Improved: Registries for Evaluating Patient Outcomes and HIT (Text Version)
On September 27, 2010, Elise Berliner, Rich Gliklich, and Nancy Dreyer made this presentation at the 2010 Annual Conference. Select to access the PowerPoint® presentation (2 MB).
New and Improved: Registries for Evaluating Patient Outcomes and HIT
Elise Berliner, Ph.D.
Director, Technology Assessment Program
Center for Outcomes and Evidence
- Introduction to the Second Edition
- Elise Berliner, Ph.D., Task Order Officer
- Review of major changes and new sections
- Richard Gliklich, M.D., Senior Editor
- Nancy Dreyer, Ph.D., M.P.H., Senior Editor
- Planning for the Third Edition
- Audience discussion, led by Dr. Berliner
- First edition, published in 2007, has been widely used as a reference for designing, operating, and evaluating patient registries.
- Numerous citations (> 60) in literature and in significant government publications (e.g., Federal Register, FCC Report to the President, etc.)
- As registries continue to evolve, many new methodological and practical issues have arisen.
Purpose of the Second Edition
- Update and expand the first edition of the User's Guide with new information gathered from recent publications and reported experiences encountered by researchers and other professionals who utilize registries for research.
- Expound on selected topics in the original guide and add new topics that deserve further in-depth discussion.
Process for Creating the Guide
- Topics identified based on public comments received for the first edition.
- New sections:
- Author and reviewer teams assembled with similar balance to first edition.
- Posted for public comment and revision.
- Original chapters:
- Original authors and reviewers were invited to participate. Additions made for new topic areas when necessary.
- Full, revised handbook posted for public comment.
- Open call for case examples.
Second Edition Collaborators
- 55 contributors from industry, academia, health plans, physician societies, and government.
- 49 invited peer reviewers, plus public comment.
- 38 case studies illustrate challenges and solutions (including 20 new examples).
- Senior Editors: R. Gliklich; N. Dreyer. Managing Editor, M. Leavy. Outcome DEcIDE Center.
Second Edition: Table of Contents
- Patient Registries (Overview)
- Planning a Registry
- Registry Design (includes Planning for the End of a Patient Registry)
- Use of Registries in Product Safety Assessment
- Data Elements for Registries
- Data Sources for Registries
- Linking Registry Data: Technical and Legal Considerations
- Principles of Registry Ethics, Data Ownership, and Privacy
- Recruiting and Retaining Participants in the Registry
- Data Collection and Quality Assurance
- Interfacing Registries With Electronic Health Records
- Adverse Event Detection, Processing, and Reporting
- Analysis and Interpretation of Registry Data To Evaluate Outcomes
- Assessing Quality
Registries for Evaluating Patient Outcomes:
A User's Guide
Richard Gliklich, M.D., Senior Editor
Nancy Dreyer, Ph.D., M.P.H., Senior Editor
Review of Major Changes
- Changes to Sections on Creating and Operating Registries:
- All chapters were updated. Some chapters had more significant additions.
- Multiple new case studies.
- Review of "Linking Registry Data.".
- Review of "Interfacing Registries with EHRs.".
- Highlighted Case Examples.
Updates to 1st Edition Chapters
- "Planning a Registry" (Chapter 2) now discusses public-private partnerships.
- "Registry Design" (Chapter 3) and "Analysis and Interpretation" (Chapter 13) were reorganized to align more closely.
- "Data Elements" (Chapter 5) was expanded to discuss new data standards.
Updates to 1st Edition Chapters
- "Principles of Registry Ethics, Data Ownership, and Privacy" (Chapter 8) addresses new legislation:
- Patient Safety and Quality Improvement Act of 2005 (PSQIA)
- Genetic Information Nondiscrimination Act of 2008 (GINA)
- Health Information Technology for Economic and Clinical Health Act (HITECH Act)
- "Adverse Event Detection, Processing, and Reporting" (Chapter 12) discusses risk evaluation and mitigation strategies (REMs).
Case Examples (new examples highlighted)
- Using Registries To Understand Rare Diseases
- Creating a Registry To Fulfill Multiple Purposes and Using a Publications Committee To Review Data Requests
- Using a Registry To Track Emerging Infectious Diseases
- Using a Collaborative Approach To Plan and Implement a Registry
- Using a Scientific Advisory Board To Support Investigator Research Projects
- Determining When To Stop an Open-Ended Registry
- Designing a Registry for a Health Technology Assessment
- Assessing the Safety of Products Used During Pregnancy
- Designing a Registry To Study Outcomes
- Analyzing Clinical Effectiveness and Comparative Effectiveness in an Observational Study
- Using a Registry To Assess Long-Term Product Safety
- Using a Registry To Monitor Long-Term Product Safety
Case Examples (new examples highlighted)
- Identifying and Responding to Adverse Events Found in a Registry Database
- Selecting Data Elements for a Registry
- Using Performance Measures To Develop a Dataset
- Developing and Validating a Patient-Administered Questionnaire
- Understanding the Needs and Goals of Registry Participants
- Using Validated Measures To Collect Patient-Reported Outcomes
- Integrating Data From Multiple Sources With Patient ID Matching
- Linking Registries at the International Level
- Linking a Procedure-Based Registry With Claims Data To Study Long-Term Outcomes
- Linking Registry Data To Examine Long-Term Survival
- Considering the Institutional Review Board Process During Registry Design
- Issues With Obtaining Informed Consent
- Building Value as a Means To Recruit Hospitals
Case Examples (new examples highlighted)
- Using Registry Tools To Recruit Sites
- Using Proactive Awareness Activities To Recruit Patients for a Pregnancy Exposure Registry
- Using Reimbursement as an Incentive for Participation
- Data Collection Challenges in Rare Disease Registries
- Managing Care and Quality Improvement for Chronic Diseases
- Developing a Performance-Linked Access System
- Using Audits To Monitor Data Quality
- Challenges in Creating Electronic Interfaces Between Registries and Electronic Health Records
- Creating a Registry Interface To Incorporate Data From Multiple Electronic Health Records
- Technical and Security Issues in Creating a Health Information Exchange
- Developing a New Model for Gathering and Reporting Adverse Events
- Using Registry Data To Evaluate Outcomes by Practice
- Using Registry Data To Study Patterns of Use and Outcomes
New Chapter: Linking Registry Data
Rationale: Increasingly, statistical methods are used to link data from multiple de-identified sources, including registries. For these projects:
- What is the risk of identifying patients by combining data from multiple registries?
- What are the legal and ethical requirements for researchers to ensure patient privacy and confidentiality?
Linking Registry Data: Overview
- Chapter is divided into 2 sections: Technical Aspects and Legal Considerations.
- Technical Aspects: What is a feasible technical approach to linking the data?
- Legal Considerations: Is the linkage legally feasible under the permissions, terms, and conditions that applied to the original compilations of each data set?
Table 10: Technical Planning Questions
Table 9: Legal Planning Questions
|1. Purpose for data linkage||
|2. Conditions under which data (plus or minus biospecimens) were originally collected.||
|4. The person or institution holding the data for the linkage||
Table 9: Legal Planning Questions (continued)
|5. The person or institution performing data linkage||
|6. Other laws or policies that may apply to data use or disclosure (release)||
|7. The terms and conditions that apply to data disclosure (release) and use under any agreement with the original source of the data||
|8. Anticipated needs for data validation and verification||
|9. Future needs for privacy protection of the data source or maintenance of data confidentiality||
|10. Anticipated future uses of the linked data||
Note: HIPAA is Health Insurance Portability and Accountability Act
IRB=institutional review board. NIH=National Institutes of Health.
Linking Registry Data: Case Example
- Goal: Study long-term patient outcomes for diagnostic cardiac catheterizations and percutaneous coronary interventions (PCI).
- Data Sources: CathPCI registry lacks long-term follow-up. Medicare data lacks detailed procedure information.
- Legal Approach: After review of HIPAA, Common Rule, project team determined that linkage required use of limited datasets only and IRB approval for linkage project.
- Technical Approach: CathPCI data were linked with Medicare data using probabilistic matching techniques.
New Chapter: Interfacing Registries with Electronic Health Records (EHRs)
Rationale: With national efforts to invest in EHRs, and to advance the evidence base in areas such as effectiveness, safety, and quality through registries and other studies, it is clear that interfacing registries with EHRs will become increasingly important to reduce data collection burden.
This chapter explores issues of interoperability and a pragmatic "building-block approach" towards a functional, open-standards based solution.
Interfacing Registries & EHRs: Overview
- Chapter covers 4 main topics:
- Role of EHRs and patient registries in health care.
- Vision of EHR-registry interoperability.
- Interoperability challenges.
- Partial and potential solutions.
Interfacing Registries & EHRs: Definitions
- EHR: an electronic record of health-related information on an individual that conforms to nationally recognized interoperability standards and that can be created, managed, and consulted by authorized clinicians and staff, across more than one health care organization.
- Individual focused
- Patient registry: an organized system that uses observational study methods to collect uniform data (clinical and other) to evaluate specified outcomes for a population defined by a particular disease, condition, or exposure, and that serves one or more predetermined scientific, clinical or policy purposes.
- Population focused
- Syntactic interoperability (communication): the ability of heterogeneous health information systems to exchange data.
- Wiring, application protocol, standard messaging protocol.
- Most easily solved.
- Semantic interoperability (content): implies that the systems understand the data that has been exchanged at the level of defined domain concepts.
- Depends on standard vocabulary and shared data elements.
- More difficult to solve; efforts include CDASH, ASTM Continuity of Care Record (CCR), HL7 Continuity of Care Document.
- Other issues: managing patient identifiers and authenticating users across multiple applications.
Partial and Potential Solutions
- We lack a single, complete interoperability model, but, several open standard components provide a reasonably good start which can be improved with further development, testing, and adoption of additional open standard building blocks.
- Signing, Privacy, Encryption
- Data standards (e.g. HL7 and CDISC)
- Content profile (e.g. CRD)
- Integration profile (e.g. RFD)
- Web services: http(s), Web browsers
- Physical network connection
Building-Block Approach: Retrieve Form for Data Capture/HITSP-TP50
Image: "Figure 4: Retrieve Form for Data Capture (RFD) Diagram" is shown. A box captioned "Form Filler" is positioned above another box captioned "Form Manager." The two boxes are connected by two lines and two downward pointing arrows labeled "Retreive Forms (ITI-a)" and "Query Form Manager (ITI-b). To the right of "Form Filler" and also connected to it by lines are a data bin captioned "Ex: Document Vault" and a box captioned "Form Receiver." Above "Ex: Document Vault" is a downward pointing arrow labeled "Archive Form (ITI-d)." Between "Ex: Document Vault" and "Form Receiver" is a downward pointing arrow labeled "Submit Form (ITI-c)."
- EHR-registry interoperability will be increasingly important as adoption of EHRs and patient registries increases significantly.
- Interoperability requires accurate and consistent data exchange and use of the information that has been exchanged.
- While a complete interoperability solution does not yet exist, enough open standard building blocks do exist today to enable 'functional interoperability' which while imperfect, significantly improves workflow and reduces duplication of effort.
Interfacing Registries & EHRs: Case Examples
- A registry focused on effectiveness in pain management was made interoperable with a commercial EHR using RFD communication.
- ASTER project: interoperability was achieved for the purpose of reporting adverse event information to the FDA.
- A commercial EHR was made interoperable with a quality reporting initiative for the American College of Rheumatology and with a Physician Quality Reporting Initiative Registry for reporting data to the Centers for Medicare and Medicaid Services.
Review of Major Changes
- Review of "Planning for the End of a Registry.".
- Review of "Use of Registries for Product Safety Assessment.".
- Highlighted Case Examples.
- Changes to Section III, Evaluating Registries.
New Section: When to End a Registry
Rationale: Registries may be developed without specific endpoints, making it difficult to determine when to end data collection. A more clear explanation of recommendations for stopping a particular registry is useful from several perspectives, including costs.
This section addresses how to determine when a registry has met its objectives and should be closed.
Stopping Decisions and Registry Goals
- Registry protocol should include specific and measurable endpoints against which to judge whether the project should continue or stop.
- Measurable goals for endpoints will make it possible to determine whether the registry has achieved a core purpose and may lead to a reasonable stopping point.
- Conversely, a registry that fails to meet measurable goals and appears to be unable to meet them in a reasonable time is also a candidate to be stopped.
- Other reasons for stopping:
- Incomplete or poor quality data.
- Registry has outlived the question it was created to answer.
- Funding and/or staffing is no longer available.
What Happens When a Registry Stops?
- Stopping a registry could mean:
- Ceasing all data collection and issuing a final report.
- Ceasing to accrue new patients while continuing to collect data on existing patients.
- Regardless, archiving rules should be checked and followed.
- No clear ethical obligation to participants to continue a registry that has outlived its scientific usefulness.
What Happens When a Registry Stops?
- Registries with one sponsor who has decided to stop the registry should be encouraged to engage other stakeholders in discussions of potential transitioning of the registry to other owners.
- Issues of data ownership, property, confidentiality, and patient privacy would need to be satisfactorily addressed to make such transitions possible.
- Reasons to consider preserving registry data:
- If data may be capable of producing a recognized public health benefit that will continue if the registry does.
- If registry has historical importance, such as a registry that tracks the outbreak of a novel infectious disease that may provide insight into the transmission of the disease if not now, then sometime in the future.
- If longitudinal data may be useful for hypothesis generation.
Ending a Registry: Case Example
- Bupropion Pregnancy Registry:
- Open-ended, observational exposure-registration and followup study to monitor prenatal exposure to bupropion and detect any major teratogenic effect.
- Registry collected data on over 1,500 exposed pregnant women over 10 years.
- In 2007, advisory committee recommended discontinuation of the registry, based on its conclusion that sufficient information had accumulated to meet the scientific objective of the registry.
New Chapter: Use of Registries for Product Safety Assessments
Rationale: Registry populations often include patients that are significantly different from patients studied in clinical trials (age, comorbidities, etc.) and therefore are of particular interest in post-approval safety monitoring.
- How should data be monitored for adverse events?
- How is the "expected" rate calculated?
- How often should the data be analyzed?
- What is the legal and ethical responsibility of the registry for reporting, informing physicians and institutions of potential problems?
Use of Registries for Product Safety Assessments: Overview
- Chapter divided into 4 sections:
- Registries specifically designed for safety assessment.
- Registries designed for purposes other than safety.
- Signal detection in registries.
- Potential obligations for registry developers in reporting safety issues.
Registries Designed for Safety Assessments
- Registries are well suited to identify effects that can only be observed in a large and diverse population over an extended period of time (drug-drug interactions, genetic differences in drug metabolism, etc.).
- When designing a registry for the purposes of safety, the size of the registry, the enrolled population, and duration of follow-up are all critical to ensure validity of the inferences made based on the data collected.
- Some other challenges:
- Recruiting a meaningful patient population.
- Evaluating the utility of a registry when the entire population-at-risk has not been included (a common scenario.)
- Understanding timing of treatments, switching of treatments, multiple therapies, dose effects, delayed effects, and patient compliance.
Registries Designed for Purposes Other than Safety
- Registries examining comparative effectiveness, the natural history of a disease, evidence in support of national coverage decisions, or quality improvement efforts may gather and report AE data, but may not be able to reliably detect all events.
- Facilitate safety reporting rather than evaluate it.
Use of Registries for Product Safety Assessment: Case Example
- British Society for Rheumatology Biologics Register (BSRBR):
- Prospective observational study to monitor the routine clinical use and long-term safety of biologics in patients with severe rheumatoid arthritis and other rheumatic conditions.
- Registry enrolls patients on anti-TNF agents and a control cohort of patients on DMARDs.
- Data from registry have been used to determine whether an increased risk of tuberculosis existed in patients treated with anti-TNF therapy. Analysis found a differential risk among the three anti-TNF agents (Dixon et al., Annal Rheum Dis 2010).
- Tables in this chapter were re-organized to group the good practices by concept.
- Research Quality: Basic Elements and Potential Enhancements of Good Practice for Establishing and Operating Registries.
- Evidence Quality: Indicators of Good Evidence Quality and Indicators of Enhanced Good Evidence Quality for Registries.
- Chapter was updated to include good practices related to the new handbook sections.
Table 20: Research Quality—Basic Elements of Good Practices for Establishing and Operating Registries
Table 21; Research Quality—Potential Enhancements to Good Practices for Establishing and Operating Registries
Table 22: Evidence Quality—Indicators of Good Evidence Quality for Registries
Table 23: Evidence Quality—Indicators of Enhanced Good Evidence Quality for Registries
|Analysis and reporting
Plan for the Third Edition
- Development begins in Fall 2010.
- Plan is to include 11 new chapters, plus case examples.
- 7 topics identified through public comments on second edition.
Topics for the Third Edition
- Patient identity management
- Protection of data from litigation
- Data protection concerns
- Public-private partnerships
- Statistical techniques for analyzing combined data
- Pregnancy registries
- Registry transitions
For discussion: What other topics should be addressed in the third edition?