Skip Navigation Archive: U.S. Department of Health and Human Services U.S. Department of Health and Human Services
Archive: Agency for Healthcare Research Quality
Archival print banner

This information is for reference purposes only. It was current when produced and may now be outdated. Archive material is no longer maintained, and some links may not work. Persons with disabilities having difficulty accessing this information should contact us at: Let us know the nature of the problem, the Web address of what you want, and your contact information.

Please go to for current information.

Using Patient Experiences Surveys in Health Plan and Practice Evaluation (Text Version)

Slide presentation from the AHRQ 2009 conference.

On September 19, 2009, Sarah Hudson Scholle made this presentation at the 2009 Annual Conference. Select to access the PowerPoint® presentation (626 KB).

Slide 1

Using Patient Experiences Surveys in Health Plan and Practice Evaluation
Sarah Hudson Scholle
Assistant Vice President, Research


Slide 2


  • NCQA
  • Health plan accreditation model and CAHPS
  • Incorporating patient experiences surveys into evaluation of physician practices


Slide 3

NCQA: A Brief Introduction

  • Private, independent non-profit
    health care quality oversight organization founded in 1990
  • Committed to measurement,
    transparency and accountability
  • Unites diverse groups around common goal: improving health care quality


Slide 4

NCQA Health Plan Accreditation

  • Key Components
    • Rigorous on-site review of key systems and processes
    • Evaluation of clinical performance through HEDIS® measures
    • Member experience surveys - CAHPS® 4.0H for adults and children


Slide 5


Clinical Performance(HEDIS) + Member Experience (CAHPS) =43%
Health Plan Systems(Accreditation Standards) = 57%

Accreditation is Performance-based:NCQA Accreditation is the only health plan Accreditation that requires reporting on clinical performance


Slide 6

What is NCQA's HEDIS?

The Healthcare Effectiveness Data and Information Set:

  • Process and outcomes measures
  • Standardized member experience surveys
  • Used by commercial, Medicare, and Medicaid plans alike
  • Allows plan-to-plan comparisons by quality, not just by price


Slide 7

CAHPS® 4.0H Surveys Development and Reporting

  • Quality Compass® (plan-to-plan comparisons)
  • State of Health Care Quality Report
  • National CAHPS Benchmarking Database (NCBD)
  • Other products-report cards, Quality Dividend Calculator, etc.


Slide 8

NCQA Recognition Programs

  • >14,000 Physicians Recognized nationality across all Recognition Programs
  • Clinical ptograms
    • Diabetes recognition program(DRP)
    • Heart.Stroke recognition program(HSRP)
    • Back pain recognition program(BPRP)
  • Medical practice process and structural measures
    • Physician practce connections
    • Physician practice connections-patient-centered medical home(PPC-PCMH)


Slide 9

Goals for Physician Practice Connections (PPC)

  • Evaluate systematic approach to delivering preventive and chronic care (Wagner Chronic Care Model)
  • Build on IOM's recommendation to shift from "blaming" individual clinicians to improving systems
  • Create measures that are actionable for physician practices
  • Validate measures by relating them to clinical performance and patient experience results


Slide 10

Theoretical Frameworks Informing Development of PPC-PCMH
Based on best available empiric evidence in each area and on testing of reliability and validity of elements in field tests using on site audit as "gold" standard

Chronic Care Model  Patient Centered Care  Cultural Competence  Medical Home 
Clinical information Systems  Respect Patient Values  Culturally competent interactions  Personal physician 
Decision Support  Accessible   Language services  Physician directed team 
Patient Self-Management  Family-Centered   Reducing disparities  Whole person orientation 
Delivery System Redesign  Continuous     Care is coordinated and integrated 
Community Linkages  Coordinated     Quality and safety 
Health Systems  Community Linkages    Enhanced access 
  Culturally Appropriate      
  Emotional Support      
  Information and Education      
  Physical Comfort     
  Quality Improvement  

First contact-comprehensive-continuous-coordinated


Slide 11

Adapting PPC for the Patient-Centered Medical Home

  • New PPC-PCMH version released in January 2008
    • Aligned standards with Joint Principles
    • Incorporated critical attributes of PCMH
    • Defined foundational elements ("must pass" requirements)
  • PPC-PCMH endorsed by ACP, AAFP, AAP, AOA, other specialties and PCPCC for use in demos

Endorsed by National Quality Forum Sept 2008 (as “Medical Home System Survey”)

Slide 12

PPC-PCMH Content and Scoring

**Must Pass Elements


Slide 13

Examples of Initiatives Using PPC-PCMH

  • Multi-payer - Colorado, Pennsylvania, Rhode Island
  • State-wide - Pennsylvania, Vermont, Maine
  • Single payer - EmblemHealth, Humana
  • Government - Medicare, New York City, Louisiana


Slide 14

Significant PPC-PCMH Issues for Future

"Measures of Meaningful Use" inside

  • How to further assess patient-centeredness, including patient survey results?
    • How to engage patients?
    • How to make name resonate positively?
  • When should performance results be part of scoring?
  • How to adapt to promote quality and cost gains across settings?
    • Primary care-subspecialty
    • Physician-hospital, other facilities
  • How to streamline requirements, documentation?
    • For all practices
    • For practices renewing


Slide 15

Timeline, Evolution of PPC-PCMH

2009 2010

Solicit input: Website, calls, meetings

Oct. - Convene Advisory Committee; develop draft changes

April -Review draft changes with CPP; Public Comment

July to Oct. - Advisory

Dec. - CPP, BOD approval


Slide 16

Barriers to Incorporating
Patient Experiences Results

  • Lack of agreement on core content
  • Whether existing tools are able to detect change in performance
  • The burden of conducting patient experiences surveys
  • Conflicting priority of accountability versus quality improvement goals
  • Structure/process versus outcome scoring


Slide 17

Aims of Proposed Research

  • Identify a core set of survey items
  • Explore feasibility of alternative sampling and other data collection strategies
  • Examine the impact of alternative scoring approaches in blending results from the PPC-PCMH and patient experience surveys


Slide 18

Getting to Core Set of Measures

  • Identify possible domains/items
    • Access
    • Communication
    • Coordination
    • Shared decision-making
    • Self Management
    • Whole person orientation
  • Ranking exercise involving broad stakeholder participation
  • Review of psychometric properties
  • Recommendations to PPC-PCMH review panel


Slide 19

Getting to Standardized Data Collection

  • Profiles of existing efforts to collect, analyze and report patient experiences survey results
    • Purpose of survey
    • Unit of Analysis
    • Sampling
    • Data collection
    • Analysis
    • Quality assurance
    • Tool
    • Cost/Finance
  • Review of literature on impact of different methods
  • Recommendations to PPC-PCMH review panel


Slide 20


  • Patient's views are critical to evaluations of health care, both at health plan and physician level
  • Standardized tools and methodology needed to allow fair, national comparisons
  • Feasibility and relevance to key stakeholders must be addressed


Slide 21

  • For more information:
  • Sarah Hudson Scholle, MPH, DrPH
  • 202-955-1726
Current as of December 2009
Internet Citation: Using Patient Experiences Surveys in Health Plan and Practice Evaluation (Text Version). December 2009. Agency for Healthcare Research and Quality, Rockville, MD.


The information on this page is archived and provided for reference purposes only.


AHRQ Advancing Excellence in Health Care