Evidence Report/Technology Assessment: Number 90
This information is for reference purposes only. It was current when produced and may now be outdated. Archive material is no longer maintained, and some links may not work. Persons with disabilities having difficulty accessing this information should contact us at: https://info.ahrq.gov. Let us know the nature of the problem, the Web address of what you want, and your contact information.
Please go to www.ahrq.gov for current information.
Under its Evidence-based Practice Program, the Agency for Healthcare Research and Quality (AHRQ) is developing scientific information for other agencies and organizations on which to base clinical guidelines, performance measures, and other quality improvement tools. Contractor institutions review all relevant scientific literature on assigned clinical care topics and produce evidence reports and technology assessments, conduct research on methodologies and the effectiveness of their implementation, and participate in technical assistance activities.
Select for PDF File (325 KB). Plugin Software Help.
Introduction / Methods / Results / Discussion / Future Research / Availability of Full Report / References
In recent years, it has become clear that the healthcare system in the United States is not providing the same quality of care for ethnic minority populations that it does for the majority white population. Racial and ethnic disparities in access to and quality of healthcare have been extensively documented.1 The Institute of Medicine report, Unequal Treatment, confirmed that racial and ethnic disparities in healthcare are not entirely explained by differences in access, clinical appropriateness, or patient preferences.2 There is also increasing evidence that provider behaviors and practice patterns contribute to disparities in care.3
Despite extensive documentation of inequities in healthcare quality, little is known about strategies with the potential to improve the quality of healthcare for ethnic minority populations. For those interested in quality improvement, there is a need for an evaluation and synthesis of the strategies that have been shown to be effective in bettering the quality of healthcare for ethnic minorities.
The purpose of this report is to systematically review the evidence to determine the effectiveness of interventions designed to improve the quality of healthcare and/or to reduce disparities for ethnic minorities. It focuses on evaluations of interventions aimed at healthcare providers or organizations, as recent work suggests these factors contribute substantially to the inequities. We examined broadly any type of strategy aimed at improving the quality of care in an ethnic minority population of patients, and then looked more specifically at strategies designed to improve the cultural competence of healthcare providers or organizations.
Return to Contents
The project consisted of engaging technical experts, formulating and refining the specific questions, performing a comprehensive literature search, reviewing the content and quality of the literature, constructing the evidence tables, synthesizing the evidence, and submitting the report for peer review.
The original questions were refined through team discussions, input from internal experts, and review and feedback from the external technical experts to arrive at the questions addressed in this report:
- What strategies targeted at healthcare providers or organizations have been shown to improve minority healthcare quality?
- Which of these strategies have been shown to be effective in reducing disparities in health or in healthcare between minority and white populations?
- What are the costs of these strategies?
- What strategies have been shown to improve the cultural competence of healthcare providers or organizations?
- What are the costs of these strategies?
We performed electronic searches of MEDLINE®, the Cochrane Collaboration's CENTRAL Register of Controlled Trials, EMBASE, and the following three specialty databases: the specialized register of Effective Practice and Organization of Care Cochrane Review Group (EPOC), the Research and Development Resource Base in Continuing Medical Education (RDRB/CME), and the Cumulative Index of Nursing and Allied Health Literature (CINAHL®). No limits were based on type of healthcare provider or specific minority group. Hand searching of key journals and reference lists was also performed. Electronic searching was completed in February 2003, and hand searching was completed to June 15, 2003.
Pairs of reviewers screened articles for eligibility at the abstract level and during review of full-text articles. Articles included in this evidence synthesis were English-language reports of original data that addressed one of the specific research questions. Specific exclusion criteria were developed in consultation with the technical experts. Articles that reported an evaluation of an intervention targeted at a healthcare provider or organization were included.
We assessed study quality and abstracted data from each eligible article. Forms for these tasks, developed in consultation with experts, were pilot tested. The strength of the evidence supporting each question was graded in relation to specific criteria through a consensus process; grades were based on quality, quantity, and consistency of the body of evidence and comprised Evidence Grade A for the best or strongest evidence, to Evidence Grade D for the weakest evidence.
Return to Contents
We screened 3,703 articles for eligibility at the abstract review level. From this screening, 288 articles were identified for full-text or article review. At this second level, 68 percent of the articles did not meet eligibility criteria. Therefore, for this report, data were synthesized from 91 eligible articles. Twenty-seven articles addressed the broad research question concerning interventions to improve healthcare quality; 64 articles addressed the specific question of strategies to improve cultural competence. Since the early 1990s there has been a striking increase in the number of articles addressing these questions; 33 percent of the 91 reviewed articles were published after 2000.
Question 1: Effectiveness of Healthcare Quality Improvement Strategies for Racial/Ethnic Minorities
Overview of Reviewed Studies
All studies were randomized controlled trials (n=20) or concurrent controlled trials (n=7). Most articles were in the area of prevention (n=19) and most targeted physicians only (n=17). The primary provider intervention was a tracking/reminder system in 10 studies, multifaceted interventions in 9, provider education in 2, bypassing the physician using nurse/nurse practitioners in 2, use of a structured patient questionnaire in 1, use of remote simultaneous translation in 1, use of subspecialty consultation in 1, and use of defibrillators on emergency medical vehicles in 1. Approximately half (n=14) of the studies had a patient intervention component, although these studies varied in whether the patient intervention was provided in addition to the provider intervention or compared with the provider intervention. The intervention was targeted to improve the quality of care specifically for racial/ethnic minorities in only two studies. The most common outcomes were related to healthcare process: appropriateness of care (n=18), quality of providers (n=9), and use of services (n=7).
Quality of Reviewed Studies
Most studies (20 of 27) clearly described healthcare providers and setting, and most (24 of 27) described the intervention sufficiently to ensure replication. Although there were 20 randomized controlled trials, the randomization was considered adequate (in that investigators could not predict assignment) in only 11 studies. Although there were seven concurrent controlled trials, there was one study in which the comparison group was considered inadequate (dissimilar).
Finally, although all studies used objective methods to evaluate outcomes, only nine of 27 had masked outcome assessment, and 13 of 27 performed a pre- and a post intervention evaluation. Approximately half (15 of 27) reported the numbers for and reasons for non-inclusion in the study analysis, and almost all (21 of 27) performed a complete statistical analysis (including the magnitude of difference between groups, an index of variability, and a test statistic).
Results of Reviewed Studies
Twenty-seven articles qualified for review, each of which used a unique combination of intervention methods in a variety of settings and patient populations. For the purpose of synthesis, we have identified the main intervention method. The categorization of the main intervention method is a simplification of what was often a complex intervention strategy.
Tracking/reminder systems. Ten studies used tracking and/or reminder systems to improve quality of care; of these studies, two were in adult general prevention,4,5 six in adult cancer screening,5-10 one in tobacco cessation,11 and one in end-of-life care (completion of advance directives).12 All ten studies demonstrated positive outcomes, primarily in the appropriateness of care (such as provision of preventive care, tobacco cessation counseling, or advance directive counseling) category. Overall, there is excellent evidence supporting the use of tracking/reminder systems aimed at providers of racial/ethnic minority patients (Evidence Grade A).
Multifaceted interventions. Nine studies used an intervention characterized as multifaceted, meaning that there were two or more main intervention methods.13-21 Two of these interventions were in adult cancer screening,13,14 one in tobacco cessation,15 one in cholesterol reduction,16 three in mental health,17-19 one in acute upper respiratory tract infections,20 and one in asthma.21 Outcomes of these studies are mixed, with most studies showing improvements in one or two (but not all) outcomes measured. Overall, there is fair evidence supporting the use of multifaceted interventions aimed at providers of racial/ethnic minority patients (Evidence Grade C).
Bypass the physician. Two studies (both in adult cancer screening) bypassed the physician and had either a nurse or a nurse practitioner offer screening directly to patients,22,23 and both studies demonstrated improvements in the provision of preventive services to patients. Overall, there is fair evidence supporting the use of bypassing the providers of racial/ethnic minority patients to offer standardized services directly to patients (Evidence Grade C).
Provider education. Two studies used provider education as the main intervention strategy, one in the area of adult general prevention24 and one in prevention of injuries in children.25 Both studies demonstrated improvements in provider counseling behaviors,24,25 but one measured and did not find any effect of the intervention on parent knowledge of injury prevention (the only outcome categorized as efficacy of treatment) or parent adherence to provider advice.25 Overall, there is fair evidence supporting the use of provider education aimed at providers of racial/ethnic minority patients (Evidence Grade C).
Use of Safe Times Questionnaire (STQ). One study (in the area of prevention in children) used a structured questionnaire to assess adolescent health behaviors and demonstrated a positive impact on providers' counseling behaviors.26 Overall, there is poor evidence supporting the use of structured questionnaires for racial/ethnic minority patients (Evidence Grade D).
Use of Remote Simultaneous Translation (RST). One study compared the accuracy of translation and quality of patient-physician communication by using remote simultaneous and proximate consecutive interpretation and found fewer translation errors and greater patient and physician satisfaction.27 Overall, there is poor evidence supporting the use of RST for racial/ethnic minority patients (Evidence Grade D).
Use of specialty consult. One study evaluated the use of nephrology consults for patients with chronic kidney disease and found no effect on health care process or patient outcomes.28 Overall, there is poor evidence supporting the use of specialty consults aimed at providers of racial/ethnic minority patients (Evidence Grade D).
Use of defibrillators on emergency medical services. One study evaluated the use of defibrillators on emergency medical services and found no effect on patient outcomes.29 Overall, there is poor evidence supporting the use of defibrillators on emergency medical services (Evidence Grade D).
Results for Question 1a: Strategies to Reduce Disparities
Only one study specifically addressed the question of whether an intervention could reduce disparities in healthcare quality between minorities and white persons.18 The study, in which two different culturally tailored interventions to improve the quality of depression care were evaluated and compared to a control group that received no intervention, had mixed results.
There was no differential effect of the interventions on healthcare process for white versus minority patients; all patients (African American, Latino, and white) in the intervention groups were more likely than patients in the control group to receive appropriate therapy. However, there was a mixed effect on health outcomes: there were improvements for African-American and Latino patients in the rate of depression compared with controls (with no improvement for white patients), whereas there were no improvements for African-American and Latino patients in the intervention groups in employment rates compared with controls (with improvement for white patients). Overall, there is poor evidence to determine which interventions might reduce disparities between racial/ethnic minority patients and majority patients (Evidence Grade D).
Results for Question 1b: Costs of Quality Improvement for Racial/Ethnic Minorities
Only one study reported on the costs of an intervention aimed at improving the quality of healthcare for racial/ethnic minority persons.28 This study, which provided case management and nephrology consultation for patients with chronic renal insufficiency, estimated a minimum yearly cost of $89,355 in 1998 (or $484 per intervention patient) and it was unable to demonstrate any health benefits in its participants. Overall, there is poor evidence to determine the cost of strategies to improve the quality of care for racial/ethnic minorities (Evidence Grade D).
Question 2: Effectiveness of Cultural Competence Training
Overview of Reviewed Studies
Of the 64 articles that qualified for our review, only two described randomized controlled trials, eight studies were concurrent controlled trials, and four had an external (non-concurrent) control group. Most studies were designed without a comparison group; these had either a postintervention evaluation only (n=25), a pre- and a postintervention evaluation (n=20), or a qualitative evaluation (n=5). Most of the interventions targeted nurses (n=32) or physicians (n=19).
The content of the curricular interventions varied. Using a previously developed framework to categorize cultural competence curricular content,30 we found that most interventions focused on specific cultural content (n=45), general concepts of culture (n=43), language (n=15), and patient-provider interaction (n=13). In terms of the specific ethnic minority groups that were the focus of the interventions, 20 studies mentioned Hispanic persons; 19, African Americans; 16, Asians/Pacific Islanders; and 5, American Indians.
Most interventions used more than one training method, and no two studies used exactly the same methods. The most common training methods were group discussion (n=29) and lectures (n=29). Most studies used more than one method for evaluation; the most common method was provider self-assessment forms (used in 33 studies). Only four articles attempted to measure patient outcomes. Most included some measure of provider outcome; attitude (n=44), knowledge (n=30), or skills/behaviors (n=22).
Quality of Reviewed Studies
Notably, less than half (n=27) of the studies had an objective outcome assessment. Only one third (n=21) included enough detail about the intervention to ensure replication. Only 17 of the interventions were developed with a theoretical model. Only 21 studies clearly described the targeted healthcare providers, setting, and dates of study. Only 15 had a complete statistical analysis. Only 14 included the numbers and reasons for non-inclusion in the study analysis. Only eight had an adequate comparison group (concurrent and similar). Only two had masking of outcome assessors, and only one had adequate randomization.
Results of Reviewed Studies
In our results below, we focus on the 34 studies with the strongest study design (studies that either had a comparison group and/or did a pre- and postintervention evaluation). We do not focus on articles that described interventions evaluated qualitatively or with only a post-test.
Knowledge. Of the 19 studies that evaluated the effect of cultural competence training on the knowledge of healthcare providers, 17 demonstrated a positive effect, one study showed no effect, and one study demonstrated a partial/mixed effect. Eleven of these studies tested the provider's knowledge about general cultural concepts, seven evaluated culture-specific knowledge, and one did not provide details to allow determination of content. There was no obvious pattern regarding which type of knowledge was enhanced by cultural competence training. Overall, there is excellent evidence to suggest that cultural competence training increases the knowledge of healthcare providers (Evidence Grade A).
Attitudes. Of the 25 studies that evaluated the effect of cultural competence training on the attitudes of healthcare providers, 21 demonstrated a positive effect, one showed no effect, and three showed a partial/mixed effect. The most common attitude outcome measured was cultural self-efficacy (measured in three studies), but other types of attitudes were greater understanding of the impact of sociocultural issues on the patient-physician relationship, more positive attitudes toward community health issues, and an increased interest in learning about patient and family backgrounds. Overall, there is good evidence to suggest that cultural competence training favorably affects the attitudes of healthcare providers (Evidence Grade B).
Skills. Of the 14 studies that evaluated the effect of cultural competence training on the skills of healthcare providers, all demonstrated a positive effect. For example, in one study, participants were given 16 one-hour sessions in which they practiced communication skills with the community volunteers. They were subsequently shown to be significantly more competent in interviewing a non-English-speaking person as rated by a masked psychologist who viewed videotapes of interviews. Other types of skills/behaviors improvements were an increase in nurses' involvement in community-based cancer education programs, an increase in self-reported social interactions with peers of different races/ethnicities, and an improved ability of participants to conduct a behavioral analysis and treatment plan. Overall, there is good evidence to suggest that cultural competence training favorably affects the skills/behaviors of healthcare providers (Evidence Grade B).
Patient outcomes. Only three articles evaluated patient outcomes: one targeted physicians,31 one targeted mental health counselors,32 and one targeted a mixed group of providers.33 All three reported favorable patient satisfaction measures,31-33 and one demonstrated improved adherence to followup among patients assigned to the intervention group providers.32
In terms of the methods used to bring about such improvements in patient satisfaction and (in one case) adherence, one study trained four mental health counselors about the attitudes that low-income, African-American women bring to counseling (4 hours total),32 another trained nine physicians to speak Spanish (20 hours total), 31 and a third implemented a state-mandated, 3-day training program focused on team training, recipient recovery principles, clinical issues, and cultural competence for all staff who have contact with recipients of inpatient mental healthcare.33 Overall, there is good evidence that cultural competence training improves patient satisfaction (Evidence Grade B) and poor evidence that it affects patient adherence or health outcomes (Evidence Grade D).
Results for Question 2a: Costs of Cultural Competence Training
Of the 55 articles eligible for review, only five addressed the costs of cultural competence training.31,34-37 Four of the five34-37 described the costs of interventions that involved international travel. In all cases students paid for some portion of the trip, while the school or program paid $0 to $2,000. There are limited data on the costs of classroom or other types of instruction. One study estimated the cost of 20 total hours of Spanish-language instruction for nine physicians to be $2,000 in 2000, not including the opportunity costs for physician time (approximately 20 hours total for each physician).31 In another program, 60 hours of classroom instruction (20 hours of Spanish-language instruction and 40 hours of cultural competence training focused on Hispanic populations) provided for 19 students had an estimated local cost of $3,000 in 1994, of which each student contributed $80.36 Finally, one program matched involved matching 26 students to 26 local ethnically diverse families, asked the students to visit the family six times, and paid each family $400 in 1996-2000.35 Overall, there is poor evidence to determine the costs of cultural competence training (Evidence Grade D).
Return to Contents
Question 1: Effectiveness of Healthcare Quality Improvement Strategies for Racial/Ethnic Minorities
There is excellent evidence that provider tracking/reminder systems are effective in improving the quality of care for racial/ethnic minority patients (Evidence Grade A), fair evidence that multifaceted interventions, provider education interventions, and interventions which bypass the physician to offer screening services to racial/ethnic minority patients can improve quality of care (Evidence Grade C), and insufficient evidence for the use of any of the studied interventions (Evidence Grade D).
Notably, however, two types of interventions had favorable results (employed in one study each, thus receiving an evidence grade of D) that may be worthy of further study:
- Use of remote simultaneous translation for patients with limited English proficiency.
- Use of the Safe Times Questionnaire for health behavior risk assessment in adolescents.
There is poor evidence to determine which strategies are most effective in reducing disparities between ethnic minority and white populations (Evidence Grade D). The only study specifically designed to do this had mixed results with improvements in only one of the two outcomes assessed.18 There is poor evidence to determine the costs of strategies to improve care and reduce disparities for ethnic minority populations (Evidence Grade D).
Question 2: Effectiveness of Cultural Competence Training
There is excellent evidence to suggest that cultural competence training can increase the knowledge of healthcare providers (Evidence Grade A), and good evidence that cultural competence training can improve the attitudes and skills of healthcare providers (Evidence Grade B). However, the studies are heterogeneous (no two studies used exactly the same intervention methods), and it is difficult to conclude which specific types of training interventions are effective in improving particular outcomes. Even within an outcome category, there is no uniformity in outcome measurement, thus making it difficult to determine which specific types of knowledge, attitudes, or skills are affected by cultural competence training.
There is good evidence from three studies to suggest that cultural competence training can favorably affects patient satisfaction (Evidence Grade B) and poor evidence that cultural competence training can affect patient adherence (Evidence Grade D), although the one study that examined patient adherence demonstrated a positive impact. There are no studies that have evaluated patient health outcomes.
There is poor evidence to determine the cost of cultural competence training (Evidence Grade D). One of the studies demonstrated an improvement in patient satisfaction also included information about cost, and so perhaps the best evidence is found in that study, which estimated a cost of $2,000 to train nine emergency department physicians in the Spanish language.31
Limitations of Report and Literature
This review was limited to reports published in English (after 1980), as our resources did not permit extensive searching of the non-English-language and "gray" literature. Consequently, publication bias is possible. However, recent work has suggested that results of reviews with these limits do not differ substantially from reviews with no such limits.38 Only studies that specifically presented data on racial/ethnic minorities were included.
Limitations of Report and Literature for Question 1
There were limited numbers of studies in each clinical category (except prevention), and few studies focused on priority conditions for which there are documented healthcare disparities (such as HIV/AIDS, cardiovascular disease, diabetes mellitus, and infant mortality). The majority of interventions (all but two) were generic improvement interventions targeted at providers of racial/ethnic minority patients; they did not necessarily target those aspects of care for which there are demonstrated disparities between minority and nonminority populations.
Some of the targeted processes of care were not evidence-based practices for any patient population (such as oral cavity exams or breast self-examinations for cancer screening) and thus would be unlikely to improve the quality of care or reduce disparities for racial/ethnic minority patients. Most studies measured health processes, rather than patient outcomes. This characteristic poses a significant limitation for studies that targeted processes of care not already linked to patient outcomes (that is, not evidence-based).
Evaluating the effectiveness of specific interventions was challenging for several reasons. Each study used slightly different intervention methods, thereby making generalizations across studies difficult. The studies used multicomponent interventions and did not examine separate components.
Very few studies involved Hispanic populations, and none included American Indians/Alaska Natives or Asians/Pacific Islanders. Most studies had no data on costs.
Only interventions targeting providers/organizations were included in this review. Although targeting patients directly may be a promising strategy to improve quality of care and reduce racial/ethnic disparities, such interventions are not reflected here. Only randomized controlled trials and concurrent controlled trials were included; there may be other worthwhile interventions that have been evaluated with other study designs.
Eligibility for our review was limited to studies in the United States, even though there may have been other promising studies conducted in other countries. Finally, we made no assessment of the generalizability of the population of providers targeted in these studies to the broader population of providers caring for racial/ethnic minorities.
Limitations of Report and Literature for Question 2
There are no standardized instruments for measuring cultural competence, and very few outcome assessments were objectively measured. Often there were no data concerning the psychometric properties of the instruments used for evaluation, and most studies were designed without a comparison group for evaluation.
Many articles did not describe the curricular interventions well enough to ensure replication. Furthermore, each curricular intervention was different, making generalizability across studies difficult.
Few studies measured patient outcomes, and none measured healthcare process quality indicators. Some studies used curriculum evaluation as the only outcome. Finally, most studies did not include data on costs.
We made no attempt to assess the psychometric properties of the instruments used to measure cultural competence. Our review focused on interventions aimed at the education of healthcare providers, rather than on an evaluation of all possible organizational strategies to provide culturally and linguistically appropriate services.
Return to Contents
Research on Improving the Quality of Care and Reducing Disparities for Racial/Ethnic Minorities
More research designed specifically to reduce demonstrated racial/ethnic disparities in healthcare quality is needed. It is necessary to distinguish between interventions aimed at improving the quality of care for all persons and those aimed specifically at improving quality of care for racial/ethnic minorities (such as reducing provider bias). More quality improvement interventions are needed that focus on priority conditions for which there are documented health disparities (e.g., infant mortality, cardiovascular disease, diabetes mellitus, and HIV/AIDS). For generic quality improvement interventions done in mixed populations, there should be subgroup analyses to gauge the effect of the interventions on equality of treatment for racial/ethnic minorities.
Several gaps in the current literature need to be filled. More studies are needed in acute care and specialty settings and also among Asian/Pacific Islander, American Indian/Alaska Native, and Hispanic populations. More information is needed about the costs of various strategies to improve healthcare quality and reduce racial/ethnic disparities. In general, studies ought to include patient outcomes, have longer followup, and link processes of care to health outcomes. There is a need to replicate promising intervention strategies in different healthcare settings and organizations.
The literature is evolving rapidly, and updated evidence assessments will be necessary soon. Funding for that research is needed.
Research on Cultural Competence
Curricular objectives need to be measurable and linked to outcomes that can be measured objectively. There is a dire need for standardized, reliable, and valid instruments to measure aspects of cultural competence. Studies should also measure the effect of the curricular interventions on healthcare process and patient outcomes. For the results to be meaningful, studies need to have a pre- and postintervention evaluation and/or a comparison group; there is certainly a need for more randomized controlled trials in this area.
Researchers should comprehensively describe the curricular interventions, such that they can be replicated in different settings. Studies also ought to include more comprehensive information about resources needed and the cost of cultural competence training.
Knowledge on this topic is evolving rapidly, and updated evidence assessments will be needed in the near future.
Return to Contents
Availability of Full Report
The full evidence report from which this summary was taken was prepared for the Agency for Healthcare Research and Quality (AHRQ) by the Johns Hopkins University Evidence-based Practice Center, Baltimore, MD, under Contract No. 290-02-0018. The full report is available from the AHRQ Publications Clearinghouse. Printed copies may be obtained free of charge by calling 800-358-9295. Requesters should ask for Evidence Report/Technology Assessment No. 90, Strategies for Improving Minority Healthcare Quality.
The Evidence Report is also online on the National Library of Medicine Bookshelf, or can be downloaded as a PDF File (1.6 MB). Plugin Software Help.
Return to Contents
1. Mayberry RM, Mili F, Ofili E. Racial and ethnic differences in access to medical care. Med Care Res Rev 2000; 57(Suppl 1):108-45.
2. Smedley BD, Stith AY, Nelson AR. Unequal Treatment: Confronting racial and ethnic disparities in health care. Committee on Understanding and Eliminating Racial and Ethnic Disparities in Health Care. Washington (DC): Institute of Medicine; 2003.
3. van Ryn M. Research on the provider contribution to race/ethnicity disparities in medical care. Med Care 2002; 40(1 Suppl):I140-51.
4. McDonald CJ, Hui SL, Smith DM, et al. Reminders to physicians from an introspective computer medical record. A two-year randomized trial. Ann Intern Med 1984; 100(1):130-8.
5. Turner BJ, Day SC, Borenstein B. A controlled trial to improve delivery of preventive care: physician or patient reminders? J Gen Intern Med 1989; 4(5):403-9.
6. Burack RC, Gimotty PA. Promoting screening mammography in inner-city settings. The sustained effectiveness of computerized reminders in a randomized controlled trial. Med Care 1997; 35(9):921-31.
7. Burack RC, Gimotty PA, George J, et al. How reminders given to patients and physicians affected pap smear use in a health maintenance organization: results of a randomized controlled trial. Cancer 1998; 82(12):2391-400.
8. Burack RC, Gimotty PA, George J, et al. The effect of patient and physician reminders on use of screening mammography in a health maintenance organization. Results of a randomized controlled trial. Cancer 1996; 78(8):1708-21.
9. Burack RC, Gimotty PA, George J, et al. Promoting screening mammography in inner-city settings: a randomized controlled trial of computerized reminders as a component of a program to facilitate mammography. Med Care 1994; 32(6):609-24.
10. Burack RC, Gimotty PA, Simon M, et al. The effect of adding Pap smear information to a mammography reminder system in an HMO: results of randomized controlled trial. Prev Med 2003; 36(5):547-54.
11. Ahluwalia LS , Gibson CA, Kenney RE, et al. Smoking status as a vital sign. J Gen Intern Med 1999; 14: 402-8.
12. Dexter PR, Wolinsky FD, Gramelspacher GP, et al. Effectiveness of computer-generated reminders for increasing discussions about advance directives and completion of advance directive forms. A randomized, controlled trial. Ann Intern Med 1998; 128(2):102-10.
13. Dietrich AJ, O'Connor GT, Keller A, et al. Cancer: improving early detection and prevention. A community practice randomised trial. BMJ 1992; 304(6828):687-91.
14. Manfredi C, Czaja R, Freels S, et al. Prescribe for health. Improving cancer screening in physician practices serving low-income and minority populations. Arch Fam Med 1998; 7(4):329-37.
15. Allen B Jr, Pederson LL, Leonard EH. Effectiveness of physicians-in-training counseling for smoking cessation in African Americans. J Natl Med Assoc 1998; 90(10):597-604.
16. Keyserling TC, Ammerman AS, Davis CE, et al. A randomized controlled trial of a physician-directed treatment program for low-income patients with high blood cholesterol: the Southeast Cholesterol Project. Arch Fam Med 1997; 6(2):135-45.
17. Burge SK, Amodei N, Elkin B, et al. An evaluation of two primary care interventions for alcohol abuse among Mexican-American patients. Addiction 1997; 92(12):1705-16.
18. Miranda J, Duan N, Sherbourne C, et al. Improving care for Minorities: Can Quality Improvement Interventions Improve Care and Outcomes for Depressed Minorites? Results of a Randomized, Controlled Trial. Health Serv Res 2003; 38(2):613-30.
19. Callahan CM, Hendrie HC, Dittus RS, et al. Improving treatment of late life depression in primary care: a randomized clinical trial. J Am Geriatr Soc 1994; 42(8):839-46.
20. Harris R, Mackenzie T, Leeman-Castillo B, et al. Optimizing antibiotic prescribing for acute respiratory tract infections in an urban urgent care clinic. J Gen Intern Med 2003; 18: 326-34.
21. Evans D, Mellins R, Lobach K, et al. Improving care for minority children with asthma: professional education in public health clinics. Pediatrics 1997; 99(2):157-64.
22. Mandelblatt J, Traxler M, Lakin P, et al. A nurse practitioner intervention to increase breast and cervical cancer screening for poor, elderly black women. J Gen Intern Med 1993; 8(4):173-8.
23. McCarthy BD, Yood MU, Bolton MB, et al. Redesigning primary care processes to improve the offering of mammography. The use of clinic protocols by nonphysicians. J Gen Intern Med 1997; 12(6):357-63.
24. Gemson DH, Ashford AR, Dickey LL, et al. Putting prevention into practice. Impact of a multifaceted physician education program on preventive services in the inner city. Arch Intern Med 1995; 155(20):2210-6.
25. Gielen AC, Wilson ME, McDonald EM, et al. Randomized trial of enhanced anticipatory guidance for injury prevention. Arch Pediatr Adolesc Med 2001; 155(1):42-9.
26. Schubiner H, Tzelepis A, Wright K, et al. The clinical utility of the Safe Times Questionnaire. J Adolesc Health 1994; 15(5):374-82.
27. Hornberger JC, Gibson CD Jr, Wood W, et al. Eliminating language barriers for non-English-speaking patients. Med Care 1996; 34(8):845-56.
28. Harris LE, Luft FC, Rudy DW, et al. Effects of multidisciplinary case management in patients with chronic renal insufficiency. Am J Med 1998; 105(6):464-71.
29. Kellermann AL, Hackman BB, Somes G, et al. Impact of first-responder defibrillation in an urban emergency medical services system. JAMA 1993; 270(14):1708-13.
30. Dolhun EP, Muñoz C, Grumbach K. Cross-cultural Education in U.S. Medical Schools: Development of an Assessment Tool. Acad Med 2003; 78(6):615-22.
31. Mazor SS, Hampers LC, Chande VT, et al. Teaching Spanish to pediatric emergency physicians: Effects on patient satisfaction. Arch Pediatr Adolesc Med 2002; 156(7):693-5.
32. Wade P, Berstein B. Culture sensitivity training and counselor's race: effects on black female client's perceptions and attrition. J Couns Psychol 1991; 38:9-15.
33. Way BB, Stone B, Schwager M, et al. Effectiveness of the New York State Office of Mental Health Core Curriculum: Direct care staff training. Psychiatr Rehabil J 2002; 25(4):398-402.
34. Godkin MA, Savageau JA. The effect of a global multiculturalism track on cultural competence of preclinical medical students. Fam Med 2001; 33(3):178-86.
35. Haq C, Rothenberg D, Gjerde C, et al. New world views: preparing physicians in training for global health work. Fam Med 2000; 32(8):566-72.
36. Nora LM, Daugherty SR, Mattis-Peterson A, et al. Improving cross-cultural skills of medical students through medical school-community partnerships. West J Med 1994; 161(2):144-7.
37. Rolls C, Inglis A, Kristy S. Study abroad programs: creating awareness of and changing attitudes to nursing, health and ways of living in other cultures. Contemp Nurse 1997; 6(3-4):152-6.
38. Egger M, Juni P, Bartlett C, et al. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technol Assess 2003; 7(1):1-76.
Return to Contents
AHRQ Publication Number 04-E008-1
Current as of January 2004