Section 7. Expert Review of Web Sites
Users of Public Reports of Hospital Quality: Who, What, Why, and How?
To better understand Web site variations in traffic and survey responses, the study investigators did an indepth qualitative review of each Web site to identify characteristics that might make it easier for visitors to process and use the quality information.
In a review of all the sites, we found that:
- Almost all the sites had fairly technical language on the top page of quality information (e.g., multiple sites refer to the prescription of angiotensin receptor blockers and use the abbreviation ARB in the reference—both the language and the acronym are overly technical for consumers).
- Many did not allow visitors to select specific hospitals to be compared next to each other.
- Most did not use composite scores to summarize areas of performance.
- Most did not use interpretive labels for performance metrics ("better," "average," "worse") to assist with interpretation of performance.
- Many had visual displays of information that were difficult to interpret. In some cases, the difficulty in interpretation arose from the data chosen for presentation, such as confidence intervals and other statistical concepts (which consumers may not understand). Other times, the site showed numeric data without either a summary number or an interpretation of what performance was best. In other cases, the difficulty stemmed from how the data were explained, such as having multiple legends on the same Web page. Another issue was using unusual approaches to allow visitors to select measures, such as arranging the measures like playing cards in a deck and leaving the burden on the user to realize that clicking on the deck leads to presentation of the next measure.
- Most Web sites presented information as individual metrics without any organizing framework (e.g., disease or condition).
- Only one Web site presented information in a conceptual framework that conveyed important elements of quality (e.g., patient experience, patient safety, delivery of care that has been shown to work) and helped visitors understand the performance ratings.
Based on this expert review, as well as the feedback from consumers, sites might consider the following recommendations to make it easier for users (particularly consumers) to compare hospitals:
- Consider making it easier (i.e., fewer clicks) to get to quality information.
- Consider allowing visitors to select hospitals to view (AHRQ has previously provided Chartered Value Exchanges [CVEs] with examples in Model Public Report Elements: A Sampler—Capacity To Narrow Selection of Providers).
- Consider allowing visitors to sort hospitals by performance (AHRQ has previously provided CVEs with examples in Model Public Report Elements: A Sampler—Capacity To Sort Providers Based on Performance).
- Consider using less technical language on the pages presenting quality information.
- Because it is hard for consumers to know what a meaningful difference is, consider interpreting the data for them. This can be done with inherently meaningful icons or visual labels that interpret the numbers, for example, excellent, good, fair, or poor performance. (AHRQ has previously provided CVEs with examples in Model Public Report Elements: A Sampler—Presentation of Measure Ratings.)
- Because consumers have an underdeveloped understanding of the larger concept of quality, use a plain language framework with no more than three or four elements or categories and report measures within those categories. This approach helps consumers comprehend more easily the meaning of individual measures as well as the larger concept of quality. (AHRQ has previously provided CVEs with examples in Model Public Report Elements: A Sampler—Presentation of Measure Ratings.)
- Consider labeling the composite measures with categories that communicate the meaning of quality: care that is proven to get results (effective); care that protects patients from harm (safe); care that is responsive to patients’ needs and preferences (patient centered). This can be done for disease-specific measures or in a more cross-cutting way (AHRQ has previously provided CVEs with examples in Model Public Report Elements: A Sampler—Presentation of Measure Ratings).
- Consider a summary page that displays summary or composite measures within the effective/safe/patient-centered framework. This page would help consumers bring the information together, help them understand the larger concept of quality, and reduce the burden of having to understand individual measures. Such a summary page could be condition specific or more general or cross-cutting (AHRQ has previously provided CVEs with examples in Model Public Report Elements: A Sampler—Presentation of Measure Ratings).