Skip Navigation Archive: U.S. Department of Health and Human Services www.hhs.gov
Archive: Agency for Healthcare Research Quality www.ahrq.gov
Archive print banner

This information is for reference purposes only. It was current when produced and may now be outdated. Archive material is no longer maintained, and some links may not work. Persons with disabilities having difficulty accessing this information should contact us at: https://info.ahrq.gov. Let us know the nature of the problem, the Web address of what you want, and your contact information.

Please go to www.ahrq.gov for current information.

The Next Revolution: The Role of Informatics in Improving Health Care

Evaluation of English and Spanish Health Information on the Internet

Slide Presentation by Gretchen Berland, M.D. (Text Version)


On August 1, 2001, Gretchen Berland, M.D., made a presentation during the Web-assisted teleconference, The Next Revolution: The Role of Informatics in Improving Health Care entitled "Evaluation of English and Spanish Health Information on the Internet."

This is the text version of Dr. Berland's slide presentation. Select to access the PowerPoint® slides (468 KB).


Slide 1

Evaluation of English and Spanish Health Information on the Internet

Gretchen Berland, M.D.
The RAND Corporation

Slide 2

Number of Persons Seeking Health Information on the Internet Has Nearly Doubled in the Last 3 Years

This slide features a line graph showing the increase in people seeking information on the Internet:

  • July 1998—54 million people.
  • June 1999—69 million people.
  • March 2001—97 million people

Slide 3

Health Information on the Internet

  • High expectations:
    • Consumers will have ready access to accurate health information.
    • Access will motivate consumers to participate more actively in their care.
  • Significant concerns:
    • Information may be incomplete, inaccurate and misleading.
  • But little is known about the accessibility, quality and reading level of that information.

Slide 4

Conducted a Three-Part Study to Evaluate English and Spanish Health Information on the Internet

  • What are consumers likely to find when they search for specific health topics online?  How easy is it to find relevant information?
  • How comprehensive, accurate and current is the information on selected e-health Web sites?
  • What is the level of literacy required to understand the information provided by these sites?

Slide 5

Overview of Approach for Studying Search Engines

  • Conditions studied: breast cancer, childhood asthma, depression, obesity.
  • Selected search engines:
    • 10 English-language.
    • 4 Spanish-language.
  • Conducted standardized searches using simple search terms.
  • Categorized the results of these searches.

Slide 6

Few Searches Lead to Relevant Information

This slide features a flow chart following an Internet search. The chart begins with entering 4 simple search terms. From there, 66% of the total links were irrelevant, and 34% were relevant. Of the relevant links, 26% resulted in a no content page, and 74% resulted in a content page. Of the content pages, 21% contained irrelevant information, and 79% contained relevant content.

Slide 7

Conducted a Three-Part Study to Evaluate English and Spanish Health Information on the Internet

  • What are consumers likely to find when they search online?  How easy is it to find relevant information?
  • How comprehensive, accurate and current is the information on selected e-health Web sites?
  • What is the level of literacy required to understand the information provided by these sites?

Slide 8

Approach Used to Evaluate Web Sites

  • Convened panels of patient advocates and clinical experts for each condition.
  • Developed 5-7 key "need-to-know" topics and "consumer-oriented" questions.
  • Developed standardized answers to questions based on literature reviews.

Slide 9

Example of Consumer-Oriented Question and Standardized Answers

"I have a lump in my breast.  What should be done to check this?"

  1. New breast lumps should be brought to the attention of a physician.
  2. Mammography and ultrasound are useful in evaluating lumps.
  3. A negative mammogram does not eliminate the need for further evaluation.
  4. A persistent, non-fluid filled breast mass felt by a physician should be biopsied.

Slide 10

Finding and Evaluating Health Information on Web Sites

  • Two searchers visited each selected Web site looking for information related to the consumer-oriented questions.
  • Results from the searches were saved and assembled into notebooks.
  • Developed a standardized rating form for each of the four medical conditions.
  • Recruited 34 physicians to rate information for both coverage and accuracy.

Slide 11

Examples of Rating Coverage: Breast Cancer Screening

No coverage:

  • No mention of mammography.

Minimal coverage:

  • Mentions mammography as a way to identify breast cancer early.
  • Does not mention who, how often, or why.
  • Does not discuss pros and cons of mammography.

More than minimal coverage:

  • Mentions who, how often, or why, or
  • Discusses pros and cons of mammography.

Slide 12

Coverage of 4 Conditions on English Sites

This slide features a bar graph of four conditions and the percentage of topics on these conditions receiving more than minimal and accurate coverage on English Web sites:

  • Breast Cancer, 63%.
  • Childhood Asthma, 36%.
  • Depression, 44%.
  • Obesity, 37%.

Slide 13

Coverage of 4 Conditions on Spanish Sites

This slide features a bar graph of four conditions and the percentage of topics on these conditions receiving more than minimal and accurate coverage on Spanish Web sites:

  • Breast Cancer, 39%.
  • Childhood Asthma, 23%.
  • Depression, 12%.
  • Obesity, 15%.

Slide 14

Conducted a Three-Part Study to Evaluate English and Spanish Health Information on the Internet.

  • What are consumers likely to find when they search online?  How easy is it to find relevant information?
  • How comprehensive and accurate is the information on selected e-health Web sites?
  • What is the level of literacy required to understand the information provided by these sites?

Slide 15

Approach to Measuring Reading Levels

  • Used widely accepted readability formulas.
  • Measured grade levels as a function of sentence and word complexity in a sample of text.
  • Applied formulas to randomly selected passages of text.

Slide 16

Health Information Not Accessible to Many

This slide features two pie charts measuring the level of education necessary to understand the information on these Web sites.

For English Language sites:

  • Middle School: 0%.
  • High School: 37%.
  • College: 52%.
  • Graduate 37%

For Spanish Language sites:

  • Middle School: 14%.
  • High School: 43%.
  • College: 43%.
  • Graduate: 0%.

Slide 17

Conclusions

  • Choice of search engine matters.
  • Overall coverage varies by language and condition.
  • Spanish-language availability and quality are lower.
  • Even if one finds information, not everyone can read it.

Slide 18

Next Steps

This slide features a diagram illustrating how the use of computers can help contribute to a health outcome. On the left are a column of four computers, two of which lead to a family, and two of which lead to a provider. The patient and provider then meet together, to produce the desired health outcome.

Slide 19

Limitations of Web Site Evaluations

  • Reviews based on content found between October-November 2000; content may have changed.
  • Reviews based on the materials that were found by searchers in 90 minutes; materials not found in this time were not evaluated.

Current as of August 2001


Internet Citation:

Berland, G. Evaluation of English and Spanish Health Information on the Internet. Slide Presentation (Text Version) presented at The Next Revolution: The Role of Informatics in Improving Health Care, Web-Assisted teleconference, August 1, 2001. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/news/ulp/informat/berlandtxt.htm


Return to Web Conference

The information on this page is archived and provided for reference purposes only.

AHRQ Advancing Excellence in Health Care