Appendix A. Additional Detail on Round 2 and 3 Interviews

Evaluation of a Learning Collaborative's Process and Effectiveness to Reduce Health Care Disparities Among Minority Populations

Round Two. The second round of interviews sought a better understanding of (1) the location of the Collaborative lead and other participating staff within each organization; (2) how top leadership viewed the Collaborative in the context of broader organizational goals and any differences in views across units of the firm; (3) the firm context with respect to gathering self-reported data on race and ethnicity and using it to assess racial and ethnic disparities among members; (4) the experience of firms participating with RAND in geocoding and surname analysis; and (5) additional feedback on firms' experience with the Collaborative to date.

We conducted 54 round two interviews with 60 people—an average of about 6 interviews per firm and at least 5 interviews with each firm except one that limited its participation to a group interview with the three staff members most closely involved with the Collaborative.1 Most firms were extremely helpful in supporting our interview requests. We were able to interview many of the staff specified in the protocol along with a few others (fewer than 5 of the 54) recommended by staff because of their perceived relevance.2 While we succeeded in interviewing many of the staff types called for in the evaluation plan, we were more consistently able to interview those most closely involved in the Collaborative than others. In addition, the executives interviewed were more likely to represent the clinical side of the organization than health plan administration. This means that while we gained insight into marketing, legal, and information technology (IT) issues, we were often not able to interview staff specifically responsible for these functions.3 In about half the firms, we succeeded in gaining access to the most senior executives, including the CEO of a national firm and several medical or operations directors. In the other half, we were told that executives viewed the staff we were interviewing (who were senior) as capable of adequately conveying their concerns. Though disappointing, the responses seem understandable given the demands on executives' time.

In addition to telephone interviews, we planned to collect two other types of data in round two. To provide feedback on communications within the network, we asked each of the nine participating firms and six support organizations/sponsors to complete a structured worksheet that elicited feedback on both the Collaborative overall and the contribution of diverse organizations. We planned to complement this with a request that firms complete a data capabilities worksheet to provide better insight into baseline and current capabilities in collecting self-reported racial and ethnic data for members. However, we dropped this component because of the potential burden and conflict with competing pressures on firms in the Collaborative,4 along with our growing recognition that most firms had only collected race and ethnicity for a very small proportion of their commercial members. (Firms can get limited racial and ethnic data for Medicare and Medicaid members directly from those purchasers.) Instead, we captured general insights into the topics covered by that worksheet as part of the interviews we conducted.

Round Three. The third round of interviews sought a better understanding of (1) how top leadership viewed the firm's commitment to disparities and their overall perspectives of the Collaborative; (2) the specific pilot initiatives carried out by the firms to address racial and ethnic disparities among their members; and (3) the experiences and perspectives of firm staff working with GMMB to address communications issues related to the work of the Collaborative. The more limited scope of inquiry in round 3—and the recognition that firms were hesitant to spend substantial time responding to questions—led to fewer interviews in round 3 than round 2.

In this round, we conducted 23 interviews with 26 people; one firm opted not to participate.5 We conducted an average of approximately three interviews per participating firm.6 We interviewed the lead contacts from all eight firms participating in the round 3 interviews. To learn about the pilot activities, we interviewed additional staff involved in the pilot initiatives from five of the firms; the other three firms either had no clear pilot activity or the lead contact said that they could provide the information we needed about it. In addition, we interviewed firm staff involved in the Collaborative's communications work at seven of the eight firms.7 Although we wanted to re-interview the same senior executives we talked with in round two, we were able to do so in only about half the firms. The others typically believed that their senior executives had provided their views earlier so talking to them again when they had many demands on their time was not a priority.


1. Of the other eight firms, we conducted five interviews with two firms, six interviews with one firm, seven interviews with three firms, and eight interviews with two firms. With one exception, we conducted fewer interviews with national firms than local/regional firms, although the overall level of cooperation among all of these firms was high.

2. Typically, staff had responsibility for diversity issues within the firm, often related to the firm's workforce objectives. As discussed in Chapter III, the interviews illustrated that, beyond their care delivery function, the firms involved in the Collaborative are major employers and, as such, have an interest in the diversity of their workforce and their ability to contribute towards broader community goals related to diversity.

3. Among the nine firms, we were able to interview the marketing designee in three firms and the legal designee in three firms. IT interviews typically took place with staff involved in geocoding and surname analysis. We were less successful in talking to staff with overall responsibility for IT within the firm. Gaps in marketing and IT interviews appear to reflect competing demands on these staff. Gaps in legal interviews reflect the difficulty of persuading firms to identify staff appropriate to respond to the legal issues related to disparities.

4. CHCS, for example, had just asked firms to respond to a survey on their capacity to provide common HEDIS measures.

5. Firms felt it was burdensome to respond to multiple requests for interviews, particularly after round 2 when substantially more interviews were requested. Also contributing to this feeling, firms typically did not distinguish between requests from the evaluation team and those from support organizations.

6. Of the other eight firms, we conducted one interview with one firm, two interviews with one firm, three interviews with four firms, and four interviews with two firms.

7. Five of the firms utilized staff from their communications/public relations divisions as the primary communications contact for the Collaborative. For one firm, the communications contact was also the staff member managing the day-to-day work related to the firms' disparities initiatives. We also found that one firm was in the process of hiring a new communications staff member (the former communications contact for the Collaborative left the firm a few months prior to our round three interviews), but the lead contact for that firm was moderately involved in the communications work and was able to speak to the firm's communications activities and perspectives.

Page last reviewed December 2007
Internet Citation: Appendix A. Additional Detail on Round 2 and 3 Interviews: Evaluation of a Learning Collaborative's Process and Effectiveness to Reduce Health Care Disparities Among Minority Populations. December 2007. Agency for Healthcare Research and Quality, Rockville, MD.