Your browser doesn't support JavaScript. Please upgrade to a modern browser or enable JavaScript in your existing browser.
Skip Navigation U.S. Department of Health and Human Services
Agency for Healthcare Research Quality
Archive print banner

This information is for reference purposes only. It was current when produced and may now be outdated. Archive material is no longer maintained, and some links may not work. Persons with disabilities having difficulty accessing this information should contact us at: Let us know the nature of the problem, the Web address of what you want, and your contact information.

Please go to for current information.

Quality Interagency Coordination (QuIC) Task Force
Return to QuIC Home
About QuIC
Steering Group
Press Releases
Related Links
Site Map
Discussion Paper

Informing Older Consumers about Health Care Quality (continued)

Decision Support in Using Quality Information

As with dissemination, we need to know more about how to provide decision support in using quality information. This is another arena in which the potential role of intermediaries is critical, but also fraught with potential controversy. Many older people just want you to tell them what they should do or choose (note this does not mean they will always do it!). On the other hand, Government-sponsored insurance counselors are carefully taught that they should not make recommendations or even share their opinions, but rather help people clarify and apply their own criteria. Putting aside the issue of how to deal with demands to make rather than support decisions, however, we have to deal with the following issues and challenges:

  • How can we help people clarify and express what is most important to them? How can we shift the "mental models" that people have today about making a decision so they incorporate attention to quality? (Select to access Educating and Motivating Consumers To Use Quality Information)
  • How can we make it easier for people to discern patterns in information, such as which plans score better across the board or in particular areas, and which ones do not?
  • What tools can we provide to intermediaries as well as older people to help them simplify decision support tasks? These might include worksheets (which have rarely been carefully studied) and decision support software, but this is an area in which innovation would be welcome.
  • Similarly, what tools can we provide to surrogate decisionmakers to maximize the potential input of the older person her/himself but also to simplify and perhaps take some stress out of the decisionmaking process?

Return to Contents

Key Issues in Developing Consumer-Relevant Quality Measures

Until this point, the primary focus of this paper has been on providing quality information to inform health plan choices, particularly choices between original Medicare and Medicare managed care, and among managed care plans. In fact, the bulk of available information addresses managed care-related choices. There is clearly a need for more and more comparable information on other health plan types. For example, information on the original Medicare program, as opposed to Medicare managed care plans, is only beginning to be available. In addition, the availability and breadth of information on quality that is available to those with retiree health benefits, with respect to options specific to their former employer, varies considerably, and is rarely comparable to that available for other options. There are non-trivial issues associated with the presentation of quality data on options which vary significantly with respect to benefits and premiums. It is difficult to identify specific measures and items that are not likely to be affected by benefits and costs (in terms of the older person's response to more objective consequences). As additional "Medicare + Choice" options come online, we will need to collect comparable data on those plans as well (e.g., private fee-for-service plans).

However, choices among plan types or specific plans may not be the health-related decisions that are most salient either to older people or their surrogate decisionmakers. In part, this is because many managed care plans insist they cannot provide information on the decision criteria they use in determining medical necessity or on the treatments and disease management protocols they use for serious illnesses. Such information might make it easier for consumers and advocates to see exactly how differences in HMO practice influence clinical care.

Indeed, in line with the sense that most older people have that it is their clinicians, and facilities such as hospitals and nursing homes, that make the greatest difference to the quality of care they get, we need to improve the supply of reliable and valid quality measures on providers rather than insurance plans. There are, of course, serious technical difficulties that have inhibited the development of such measures. While it may be technically possible to do what most older people want most-provide reliable information to compare individual clinicians, current computerized information systems make acquiring and processing such data prohibitively expensive. We need to "push the envelope" with respect to information on hospitals, medical groups, nursing homes, home health agencies, and integrated delivery systems, as well as work to support efforts to make the information-intensive world of health care embrace computerized information systems more fully.

However, in expanding the availability of quality measures, we risk making it even more difficult to keep older people and surrogate decisionmakers from becoming completely overloaded with and giving up on, quality information. A longer-term solution is needed that generates a parsimonious set of quality measures that tell a lot about the fundamental day-to-day operation either of an insurance plan or a provider entity. One way to do this is to create "indices" that summarize even more than topic-specific composites. Like many solutions, this one is somewhat controversial, since it is possible that both perceived meaning for consumers and actual variation would be lost in the aggregation process. Another approach, which will take more basic research and analysis, is to identify those measures that:

  • Correlate well with performance on many other measures.
  • Are fairly resistant to "gaming" by plans or providers.

Finally, we currently have a serious discontinuity between the people to whom quality data are most relevant and essential (people with serious illnesses and/or disabilities) and the source or character of the information available. We have a lot of information about the experiences of consumers who don't have a lot of experience; this is particularly true for consumer survey-based information. Such data may well be perceived by those with serious illnesses or disabilities as irrelevant at best or biased at worst. We need to study the effect of:

  • Oversampling in consumer surveys to reach enough people with serious illnesses and disabilities to make reliable estimates.
  • Sharing this information both with the "average" older consumer and with those particularly concerned about health care quality because of their own health.

The common wisdom in the field has been that people want information from "people like them." However, we may need to do a little persuasion so the "average" older consumer sees the potential, if not the current relevance, of the opinions of those who use health care more frequently and face more serious consequences if their health care is of poor quality.

Return to Contents

Key Issues in Identifying the Market Characteristics and Purchasing Strategies that Support Consumer Use of Quality Information

As already noted, the choice context of older people is extremely complex. Understanding the basic Medicare program is to some (probably unknown) extent a precursor to understanding the other options. The particular set of options available varies not only from market to market but among individuals within a market, depending on their income, current and former employment, and other factors such as the presence or absence of a State pharmaceutical assistance program. As noted, the number and kind of options has also been changing over time, creating both crises for those immediately affected and wariness in those watching from the wings. The more volatile markets become, the more likely it is that some of the quality information we collect and provide will be outdated, irrelevant, and of uncertain value.

However, one advantage is that since the Medicare program covers almost all people aged 65 and older, changes in Medicare can be conceived that would reduce the inherent complexity of the Medicare program, which currently makes it very difficult to educate or inform older consumers. Most of these changes would require congressional action, which means they must be fairly long-term goals, since the primary focus of most Medicare reform efforts at present is on providing more outpatient prescription coverage and addressing concerns about future financial crises.

Nevertheless, some policy changes to pursue would include, for example:

  • Eliminating the distinction, from the perspective of the person on Medicare if not for the financing of the program, between Part A and Part B.
  • Removing the limits on hospital stays and the "benefit period" concept, which probably saves the program very little money at this point in time.
  • Simplifying the coverage of nursing home stays, home health coverage, and such items as blood and ambulances.

Further, HCFA has considered already, and should perhaps conduct a formal experiment on the consequences of, standardizing the benefit packages of Medicare managed care options (in particular HMOs) in the same way that Medigap policies have been standardized to make it easier for consumers to compare both premium price and quality "on a level playing field" of coverage.

Congressional policy has emphasized that people on Medicare want and deserve more choices. Unfortunately, however, the emphasis has been on choices that are not particularly salient to most older people (choices among types of health plans) rather than choices that are of immediate concern (choices among clinicians and facilities). In the process, policymakers have offered choices that are difficult even for policy wonks to understand, such as private fee-for-service plans and medical savings accounts.

In addition, the more plan type choices are created, the more time and resources are needed to develop and implement quality measures for each new choice, and the more cognitive burden is placed on already confused consumers. When we offer choices that are only marginally different, educational efforts are sidetracked from more significant messages and data, and the average older consumer begins (with cause) to think we are more than a little out of touch with their reality.

It is also essential for HCFA to work more closely, on both a policy and operational level, with the other "purchasers" for this population, including the Medicaid program at the Federal and State levels, and private and public sector employers who provide retiree health benefits.* We need to clarify the choice context for those whose coverage is sponsored by these other organizations, and probably develop materials tailored to their needs. As we frame key messages and dissemination strategies, we need to take the complexity of this choice environment into consideration.

Another critical element of the market has a significant influence on our work educating older consumers about quality—prices are only barely beginning to be risk adjusted—in ways that even the developers of adjusters recognize are inadequate.54 Currently, absent effective risk adjusters, plans are rightly concerned about "advertising" that they are effective at taking care of people with serious health conditions or disabilities. They would likely resist the notion, articulated above, that we should oversample such people in our surveys and present the data on their responses to the system. At the same time, some at least of the current market volatility arises from the financial impact of risk adjustment on Medicare HMOs that have attracted a more healthy population than remains in original Medicare. On balance, however, we need a set of health plans and providers for older people that are willing to provide high quality care not just to the relatively healthy, but to the very, very sick.

* The partnership between HCFA and the Federal Office of Personnel Management, as well as the Agency for Healthcare Research and Quality (AHRQ), in supporting the Interagency Work Group on Consumer Information, is an excellent example of such collective action.

Return to Contents

Key Issues in Evaluating the Utility and Impact of Consumer Information Efforts

In this paper, we have noted the need for innovation and experimentation, as well as some basic research, in order to improve our performance in disseminating and presenting quality information to older consumers. To learn from our innovation and experimentation, we also need to evaluate the utility and impact of consumer information efforts. Here are some issues that must be addressed in this area of the research and action agenda:

What are the most critical evaluation questions? One way to define "most critical" is in terms of whether questions speak to the greatest and most significant "uncertainties" we have about our work.

One method for surfacing these uncertain assumptions is to develop what Patton calls a "theory of action."55 Theories of action specify in some detail the assumptions we have about how an intervention achieves its objectives. Sometimes called "logic models," they lay out, in a more or less causal sequence, how we expect to move through the steps of a consumer information initiative. In some cases, our assumptions are well grounded in evidence; in many cases, they are not. In some cases, we are taking fairly small steps from what we already know to its application in an intervention. In other cases, we are making leaps of faith over chasms of uncertainty. Developing a theory of action is both a discipline that tests the underlying strength of an intervention's design and a way of identifying where we are most uncertain about what will happen.

One useful element of a theory of action is the specification of a whole series of intermediate outcomes that have to be achieved before we can expect progress on our major long-term goals. The theory can help us focus on the right set of outcomes, at the right time, thus avoiding the premature assessment of long-term outcomes, and in particular outcomes that relate to the impact of information on decisions. Given how extremely unfamiliar consumers are with quality information, it is absurd to expect that they will respond to the "first dose" of information they receive. To use a pharmaceutical metaphor, we need to think of consumer information initiatives in terms of a medication that has to be taken for weeks before it reaches therapeutic levels, as opposed to a pair of aspirin that are expected to get rid of a headache in less than an hour.

One positive hallmark of this field has been the extensive use of formative, as opposed to summative, evaluation. Formative evaluation, like the focus groups and cognitive tests used to assess educational materials before they are finalized, helps inform the design of a program before it is carved in stone. Summative evaluation looks more retrospectively at the worth or value of a program that is fairly stable and fully developed. In fact, it may be useful to think of our evaluative inquiries as being along a continuum that depends on the stage of development of the field as a whole (i.e., the pre-existing knowledge base) and of the particular program.

In addition to the formative studies we already do, we can learn a lot from:

  • Systematic needs assessments (for example, to help design a training program for intermediaries).
  • Feasibility studies (for example, to identify which agencies will and will not be able to willing to serve particular intermediary roles).
  • Laboratory studies (to identify, for example, the level of difference in scores that makes people sit up and take notice).
  • Process evaluations (that assess the extent to which a program has actually been implemented sufficiently to have any outcomes).

We also need to distinguish between evaluations of dissemination strategies and those of educational materials or products. If the dissemination never happened, it would be foolish to see whether or not the materials to be disseminated had the desired short-, mid-, or long-term effects. Evaluators may need to work toward a consensus regarding when it is most likely that we will see various intermediate and ultimate outcomes of health quality information initiatives. As part of this process, we may need to recognize and assess outcomes related to the increase in the public's sense of accountability of the health care system, as well as outcomes related to improved decisionmaking.

Not everyone who undertakes a consumer information initiative will be in a position to conduct a systematic evaluation of it. This is all the more reason to make sure we get the most from the evaluations that are conducted. In particular, we need to work toward certain kinds of standardization, to make sure that our results can be aggregated in a meaningful way. This means moving toward the use of more standardization, for example:

  • In methods for conducting and reporting on focus groups and cognitive tests.
  • In concepts, variables, and measures used in both process and outcome studies.
  • In environmental conditions and individual characteristics on which data are needed so we know to what circumstances and persons our results can be generalized.

These steps would make it more feasible to compare interventions, and in the long run do meta-evaluations or meta-analyses.

Last but not least, we must note the need for more basic and evaluative research on the issues surrounding interventions to support family members, friends, and other surrogate decisionmakers, and the need to motivate, train, and support various kinds of intermediaries.

Return to Contents

Appendix A. Continuum of Intermediary Relationships with People on Medicare

  1. Family member or other person/agency with legal authority to make decisions on behalf of a person on Medicare.
  2. Family member or trusted friend without legal authority.
  3. Professional advisor paid by the person on Medicare.
    For example: financial planner, attorney, accountant, fee-for-service geriatric case manager.
  4. Health care professional who provides clinical services to the person on Medicare.
    For example: primary care physician; nurse; medical social worker; psychologist or other mental health professional; home health workers, both professional and non-professional.
  5. Health care provider or facility administrative staff.
    For example: admitting, billing, or discharge planning staff of hospitals, nursing homes, rehabilitation facilities, or home health agencies.
  6. Disease- or condition-specific patient support organizations.
    For example: staff and volunteers of national and local organizations such as the American Lung Association, the Alzheimer's Association, or United Cerebral Palsy; hotline operators for these organizations; informal support groups for persons with specific diseases, or their family members/caregivers.
  7. Human services professional who provides non-medical services to the person on Medicare.
    For example: caseworker/case manager; eligibility worker for services to older people or people with disabilities; social services or senior center agency staff and volunteers; legal services worker; housing support staff; librarian; adult education specialist.
  8. Designated health insurance counselor.
    For example: SHIP staff member; SHIP volunteer; hotline operator; benefits counselor or employee assistance staff of former or current employer or labor union*; staff member or volunteer at health or human services agency specifically designated and perhaps trained to provide this service.
  9. National advocacy and/or service organizations serving the needs of older/retired persons and/or people with disabilities and their state and local affiliates.
    For example: AARP, National Council on Aging, National Senior Citizens Councils, National Association of Retired Federal Employees, ADAPT.
  10. Health insurance plans.
    These include both Medicare-sponsored and employer- or union- sponsored managed care plans, and insurance companies who offer Medigap policies, employer or union sponsored indemnity supplements to Medicare, or medical savings accounts.
  11. Designers and operators of health information Web sites.
    Possibly, these include not only the HCFA Web site, but Web sites of Medicare health plans and formal and informal "chat groups" for people facing or interested in a specific health problem.
  12. Print and electronic media.
    These include both national and local, general or population specific, outlets such as newspapers, magazines, radio, TV, etc.

* Note that some employers are providing "eldercare" services to their active employees to assist them in handling issues they face with their parents or other older relatives.

Return to Contents


[Editor's Note: References are presented "as is" and will be updated in the conference monograph.]

1. U.S. Census Bureau, International Data Base, Population Summary for United States, Midyear Population, by Age and Sex: 2000 and 2025.
2. source cited as U.S. Census Bureau "Population Projections of the United States by Age, Sex, Race and Hispanic Origin: 1995-2050" Current Population Reports, p. 25-1130.
3. source cited as U.S. Census Bureau "Marital Status and Living Arrangements: March 1998 (Update)," Current Population Reports, PPL-100.
4. source cited as U.S. Census Bureau "Household and Family Characteristics: March 1998," Current Population Reports, P20-515."
5. no source cited.
6. source cited as U.S. Census Bureau "Consumer Income," Current Population Reports, issued September 1999, P60-206.
7. source cited as U.S. Census Bureau "Poverty in the United States: 1998, " Current Population Reports, issued September, 1999, P60-207
8. source cited as U.S. Census Bureau "Educational Attainment in the United States: March 1998," Current Population Reports, P20-513.
9. source cited as U.S. Census Bureau "Internet releases" and source cited as Resident Population Estimates of the United States by Sex, Race, and Hispanic Origin, April 1, 1990 to July 1, 1999, with Short-term projection to September 1, 2000. Internet release date October 25, 2000.
10. U.S. Census Bureau, Population Projections Program, (NP-D1-A) Projections of the Resident Population by Age, Sex, Race, and Hispanic Origin: 1999 to 2100, Washington, D.C., January 13, 2000.
11. Brown, H. R. Prisuta, B. Jabos, and A. Campbell, Executive Summary of Literacy of Older Adults in America (no date given); Source cited as 1992 National Adult Literacy Survey, National Center for Education Statistics.
12. Parker, RM. What is health care literacy? Presented at: Health Literacy: A National Conference; June 3, 1997; Washington, D.C.
13. Williams, MV, Parker RM, Baker DW, et al. Inadequate functional health literacy among patients at two public hospitals. JAMA 1995;274:1677-82.
14. Jackson RH, Davis TC, Murphy P, Bairnsfather LE, George RB. Reading deficiencies in older patients. Am J Med Sci 1994;308:79-82.
15. Davis TC, Arnold C, Berkel HJ, Nancy I, Jackson RH, Glass J. Knowledge and attitude on screening mammography among low-literate, low-income women. Cancer 1996;78:1912-20.
16. See for example, Kail R, Salthouse TA. Processing Speed as a mental capacity. Acta Psychol 1994 Aug; 86(2-3):199-225.
17. National Center for Health Statistics Health and Aging Chartbook from Health United States, 1999. Washington, DC, 2000.
18. Hibbard JH., Sofaer S and Jewett JJ. Condition-Specific Performance Information: Assessing Salience, Comprehension and Approaches for Communicating Quality Health Care Financing Review, Fall, 1996;18(1):95-110.
19. McGee J, Sofaer S and Kreling B, Final Report: Findings from Focus Groups Conducted for the National Committee for Quality Assurance Medicare and Medicaid Consumer Information Projects, July 1996
20. Hibbard JH, Jewett JJ, Tusler M, Sofaer S and Sasso B, Final Report: Consumer Perspectives on the Breast Cancer Quality Measures for the Foundation for Accountability, June 1996
21. National Center for Health Statistics Health and Aging Chartbook from Health United States, 1999. Washington, DC, 2000.
22. National Center for Health Statistics Health and Aging Chartbook from Health United States, 1999. Washington, DC, 2000.
23. Source cited as 1990 US Census.
24. No source cited.
25. Kelman HR, Thomas C, Kennedy GJ, Cheng J Cognitive Impairment and Mortality in older community residents. Am J Public Health 1994 Aug; 84(8):1255-60.
26. Perkins P, Annegers JF, Doody RS, Cooke N, Aday L, Vernon SW Incidence and prevalence of dementia in a multiethnic cohort of municipal retirees. Neurology 1997 Jul;49(1):44-50.
27. Gallo JJ, Lebowitz BD The epidemiology of common late-life mental disorders in the community: themes for the new century. Psychiatr Serv 1999 Sep;50(9):1158-66.
28. Rentz JO. Forecasting the effects of an aging population on product consumption: an age-period-cohort framework. Journal of Marketing Research 1991 28(3):355-60.
29. Mertz B, Stephens, N. Marketing to older American consumers. International Journal of Aging and Human Development 1986; 23(1):47-58.
30. Bernhardt KL. Consumer Problems and Complaint Actions of Older Americans: A National View. Journal of Retailing. 1981 57(3):107-23.
31. Schiffman LG and Sherman E. Value Orientations of New-Age Elderly: The Coming of an Ageless Market. Journal of Business Research. 1991 22:187-194.
32. Turk-Charles S, Meyerowitz BE, Gatz M. Age differences in information-seeking among cancer patients. Int J Aging Hum Dev 1997;45(2):85-98.
33. Klippel RE, Sweeney, TW. The use of information sources by the aged consumer. Gerontologist 1974 Apr; 14(2):163-66.
34. National Center for Health Statistics Health and Aging Chartbook from Health United States, 1999. Washington, DC, 2000.
35. National Alliance for Caregiving. The Caregiving Boom: Baby Boomer Women Giving Care. September, 1998.
36. Wright, G and Howe D "AARP Research on Caregivers and Medicare Information" Presentation at AARP Conference, January 26-27,2000.
37. Becket, NK and Pearson, ML. Report on the Support Systems Available to and Utilize by the Frail and Oldest Elderly. RAND Corporation, Santa Monica, CA. 1999.
38. Source cited as Health Care Financing Administration Focus Group Summary and Report on Medicare Beneficiary Information Resources in Orange County, CA and Philadelphia, PA, December, 1997.
39. Health Care Financing Administration, Medicare & You Fact Sheet, Issue 1, August, 1999. Baltimore, MD.
40. Kreling, B., S. Sofaer and E. Kenney "Report on Focus Groups with Friends and Family Members of People on Medicare." Report to the Center for Beneficiary Services, Health Care Financing Administration, School of Public Affairs, Baruch College, May, 2000
41. no source cited.
42.,1323,5901_333251,00.html: Source cited a survey conducted for AARP by Market Facts, Inc. between February 3 and February 27, 2000.
43.,1323,5901_448131,00.html: Source cited as International Data Corporation Internet Commerce Market Model.
44. Sofaer, S. Satisfaction and Quality in Managed Care for Older People 1998
Generations, 22(2):25-30.
45. McCall N, Rice T, Sangl J. Consumer knowledge of Medicare and supplemental health insurance benefits. Health Serv Res. 1986 Feb;20(6 Pt 1):633-57.
46. Cafferata GL. Knowledge of their health insurance coverage by the elderly. Med Care 1984 Sep;22(9):835-47.
47. Marquis, MS. Consumer's knowledge about their health insurance coverage. Health Care Financ Rev 1983 Fall;5(1):65-80.
48. Hibbard JH, Jewett JJ, Engelmann S, Tusler M. Can Medicare beneficiaries make informed choices? Health Affairs 1998 Nov-Dec; 17(6):181-93.
49. Hibbard JH, Harris-Kojetin, L, Mullin, P, Lubalin, J and Garfinkel, S. Increasing the impact of health plan report cards by addressing consumers' concerns. Health Affairs 2000 Sept/Oct;19(5):138-43.
50. Kopfman JE, Smith SW, Ah-Yun JK, Hodges A. Affective and cognitive reactions to narrative versus statistical evidence organ donation messages. Journal of Applied Communication Research 1998 Aug: 26(3):279-300.
51. Sofaer S. "A Classification Scheme of Individuals and Agencies who Serve as Information Intermediaries for People on Medicare" Report to the Center for Beneficiary Services, Health Care Financing Administration, School of Public Affairs, Baruch College, May, 2000
52. Stevens B and Smith K. Finding the usual and unusual suspects: who is providing Medicare information at the community level? Presentation to the Study Group on Consumer Information and Decision Making, Committee for Health Services Research, Medical Care Section, American Public Health Association, Boston, MA, November 12, 2000.
53. Sofaer S and Dewart T. The feasibility of involving community based organizations in providing information and support to people on Medicare. Presentation to the Study Group on Consumer Information and Decision Making, Committee for Health Services Research, Medical Care Section, American Public Health Association, Boston, MA, November 12, 2000.
54. Iezzoni LI, Ayanian JZ, Bates DW, Burstin HR Paying more fairly for Medicare capitated care. N Engl J Med 1998 Dec 24;339(26):1933-38.
55. Patton, MQ Utilization Focused Evaluation: The New Century Text, Third Edition, 1997 Sage Publications, Newbury Park, CA.

Return to Contents

Current as of October 2000

Return to Informing Consumers About Health Care Quality
QuIC Home Page
Department of Health and Human Services

The information on this page is archived and provided for reference purposes only.


AHRQ Advancing Excellence in Health Care