Skip Navigation Archive: U.S. Department of Health and Human Services
Archive: Agency for Healthcare Research Quality
Archive print banner

This information is for reference purposes only. It was current when produced and may now be outdated. Archive material is no longer maintained, and some links may not work. Persons with disabilities having difficulty accessing this information should contact us at: Let us know the nature of the problem, the Web address of what you want, and your contact information.

Please go to for current information.

Buy-Right for Health Care Quality: Evidence and Indicators—Transcript of Web Conference (continued)

Penny Daniels: Andy can you give us a brief example of how you're using the data to improve performance?

Andrew Webber: Sure, and it's not the National Business Coalition on Health-it's our members, we support our members. So what I'd like to do is give you a brief example of one of our members. This is the Alliance organization in Madison, Wisconsin. They're a very interesting coalition because they do direct purchasing, they actually, on behalf of self-funded employers, organize a healthcare plan, if you will, and do direct contracting with doctors and hospitals. They, a number of years ago, made a real commitment to produce a public report card for hospitals in their local market. And working with an innovative researcher, Judy Hibbard (ph) and committed to this notion that we've got to take this complex information and make it resonate with consumers, they worked on a design of a public report card which again, connects in particular to individual consumers. And as you can see from the slide, they developed some very simple symbols with a plus sign representing fewer mistakes, complications and deaths than expected; no sign-average number of mistakes, complications and deaths, and a minus sign, I think people can relate to that-I certainly can, more mistakes, complications and deaths than expected across some major areas-surgery, non-surgery, hip, knee, cardiac care and maternity.

Penny Daniels: So what were the results, Andy? What happened when you reported on the quality of these hospitals?

Andrew Webber: Yes, let me tell you very briefly some of the results. And the takeaway message from this is, public reporting does accelerate the process of quality improvement. That's the take home. But let me give you a little of the design, because it was very interesting. The Alliance is a coalition in Madison, Wisconsin, so they had a database across the entire state. In fact some of the measures they used were not dissimilar from what Denise talked about with the Quality Indicators, in fact, I think there's some overlap. So only the hospitals in their own market, in Madison, Wisconsin, did they make the information public. And a big attempt to get it into the press, into the media, out to employers who could then present it to their individual members. And then with the other hospitals around the state, half of them got no reports at all, and then half of them just got private reports-it didn't go public, but here is your information, here is your profile. And let me share with you some of the results.

Starting with the percentage of hospitals who had poor scores at baseline, who improved their scores in the post-report period-and here you can see the first column, the public reporting hospitals, 88% of hospitals improved their performance over time. That is twice the rate that we see in the private reports, and in the no reports. Secondly, looking at an individual measure, this is a percent of hospitals with significant improvements or declines in OB performance in the post-report period. Again, we see the public reporting group doing much better, significant increases in performance, low rates of decline, and again very little difference between the private reports and no reports.

Penny Daniels: Andy, healthcare costs have been a major business expense for a long time now. Why do you think it's taken so long to develop these kinds of value based purchasing approaches?

Andrew Webber: Well, to be honest with you, I've got to start and say that the employer community has been timid as a purchaser of healthcare. Some of that is that we've lacked the corporate leadership and commitment. Some of it relates that we don't understand well the unique dynamics of the healthcare delivery system itself. The economics of the healthcare marketplace are very different from the economics that business people find in other markets. But fundamentally I think employers have not had the tools that they've needed to implement the value based purchasing agenda. And it really does start, I think, with performance measurement. We need the features of performance measurement to be validated by experts, that's why we're so excited by the AHRQ indicators. We need to rely on available data-another reason we're excited about the AHRQ indicators. It's based on administrative data-it would be very costly to go access the rich clinical data from medical records. So I think this is a cost-effective way to produce information. Very importantly it's risk-adjusted information as we've heard for the last 20 years from the provider community. Their patient cohorts and populations are different, and particularly when you're producing mortality data, that information better be risk-adjusted. So again, we applaud AHRQ and the Quality Indicators for including that in their measurement set.

Penny Daniels: So just to summarize Andy, when you meet with business leaders to talk about this whole concept, what are the messages that you really try to stress?

Andrew Webber: Be courageous! It's time to step up to the plate. It's time to realize that you are the ultimate purchaser of healthcare. It's time to realize that you are both part of the problem in not moving this value based purchasing agenda forward, and you're part of the solution because you can make it happen as the ultimate purchaser. And some other messages that we like to leave with them, certainly related to our discussion today, is that we'll never get to the perfect measurement set and we need to realize that. So don't let the perfect be the enemy of the good. Again, the employer community has to drive this value based purchasing agenda. Public reporting, we are absolutely convinced and there is a lot of research out there, standardized measures can lead to improved quality improvement. If you make the information public, providers are concerned about their reputation, and it is going to have an impact on the investments that they need to make in quality improvement to make it sing. Again, let me end by applauding the leadership of AHRQ, they're to be congratulated for this very important step, and we're very excited, our group, in working with Denise at an upcoming seminar that we're having at our national conference. Now that we've sort of removed the label, now the hard work begins in terms of how to get that information out to the public. And so moving from the consensus of the value to implementation of these Quality Indicators, the support of public accountability and value based purchasing strategy is absolutely vital, and we're committed to making that happen over time.

Penny Daniels: Andy, thank you very much. Very nice transition into Debra Ness. Debra how interested do you think consumers are in healthcare quality information?

Debra Ness: Well, first of all, Penny, I want to add my thanks to Andy's for the work that AHRQ has done on these Quality Indicators. I think this is a really significant step forward in our quest to improve quality, and I also thank you for letting me be part of this conversation. I can't emphasize enough the importance and the urgency that exists for consumers to have this information. Every day consumers are making critical decisions that affect their health and their well-being, sometimes life and death decisions. And they're doing it with very little, if any information, to guide them in that process. And when you think about what we know about the huge variations in quality of care, the huge numbers of safety problems, it really emphasizes how important this information is. And I would reinforce what Andy said, we have a great deal of urgency for this information and we can't let the perfect be the enemy of the good.

So when I think about all the factors that are contributing to increasing consumers' interest in this information, one of the things that we all know is that our healthcare system is moving toward more patient-centered care. And you can't have that if consumers don't have the information they need to ask the right questions and to make decisions. And also, we're at a time when the way in which consumers get their healthcare benefits is changing dramatically. There's more focus on consumer directed healthcare, which means not only are consumers bearing more of the costs of healthcare, but they're also having to make more of the decisions. And when you put those two factors together you create huge incentives for information to guide those decisions. Consumers are going to begin to ask, 'What am I getting for my healthcare dollars?' They already want to know where to go to get the best care. With the change in benefit design, I think this kind of information that allows them to discern differences in quality is more important than ever.

Penny Daniels: Debra, you're obviously an advocate for public reporting, do you think it's becoming easier now for consumers to get the healthcare quality information that they need?

Debra Ness: Well, that's really a glass half-full, glass half-empty question, I think. For sure there's a lot more information out there. Andy and others have already talked about some of the fabulous initiatives that are going on right now to collect this information. I think we're seeing it happen throughout the states. Different employers, plans-there's more of this information available on the Web for consumers, but it's still for the most part, difficult for consumers to use a lot of the information that's out there. And many consumers don't even know that it exists yet.

Penny Daniels: What factors do you think are making it hard for consumers to use this information that's out there?

Debra Ness: Well, the first thing I think is the fact that with all of these new initiatives evolving, there's not yet a lot of consistency. And if you don't have consistency you have confusion. And so one of the things we have to work toward is more standardized information, more consistency around what we're measuring, and how we present that information. There are lots of different dimensions of quality. There are different things you can measure, there are different ways you can measure it, and there are big differences in what kinds of things are meaningful to consumers, more or less meaningful. Somebody mentioned earlier that there are probably many technical measures that would be important for providers from the quality improvement perspective, but may not be very useful to a consumer. So we need to always keep in mind the consumer as an end user, and make sure that the dimensions of quality that we're measuring are going to be meaningful to consumers. We also have to work harder at making the ways in which this information gets displayed more accessible for consumers. And to that end there's some good research going on right now-Andy mentioned the work of Judy Hibbard (ph), I think there's more and more understanding of what it takes to present this information in a way that consumers will understand and find useful. So I can't emphasize enough that we need to get to a place where we have consistent measures being used to present information in consistent ways so that consumers can actually understand it and compare it.

Penny Daniels: Do you have any specifics on what can be done now to make it easier for consumers to use this healthcare quality information?

Debra Ness: Well, I think the work that AHRQ has done with these Quality Indicators is a very important step forward. I think the work of the National Quality Forum is very important in this regard. That is an organization that is dedicated to establishing national standards so that we can have consistent measures that can be compared. And I think there's very good news. As a member of the National Quality Forum board, I can say that the board just voted to put these AHRQ Quality Indicators through and expedite a consensus process, which I think will result in much more widespread adoption of these measures, and therefore more consistent information out there for consumers. I already talked about the importance of the format of this information being made more consistent. We're beginning to learn how to do that. I look forward to a day when it's as easy for consumers to understand the information about healthcare quality as it is when they go to Consumer Reports and they look for information about appliances or cars. And finally, I would hope that we really keep in mind the fact that the consumer is the end user, and the measures that we use have got to be meaningful to consumers at the end of the day. And a lot of these quality indicators that AHRQ has put forward are exactly that.

Penny Daniels: A couple of minutes ago Andy talked about the Quality Counts Initiative, where it seems the public reporting was an important driver of quality improvement. What conclusions do you draw from this study?

Debra Ness: Well, I think there's a simple mantra that I repeat to myself over and over again, and that is measure, report and reward. We know that what gets measured gets improved, and as Andy just also so clearly presented, what gets measured and reported improves even more dramatically and more quickly. And so, the real emphasis has to be on the public reporting of information. Once we are getting that information out there, and into the hands of consumers, who are the ultimate decisionmakers, I have no doubt that that will help shape their decisions and be an added driver in our quest for better quality.

Penny Daniels: It sounds like you think that consumers can really change the American healthcare system.

Debra Ness: I absolutely think they can, with the right information, but they can't do it alone. And I think we all have to play a role in helping consumers to really use the power that they have. First of all, we need to make sure that purchasers are making this information available to consumers by virtue of how they contract for benefits and the kind of information that they ask providers to make available to their employees. We also need for purchasers to put the right kinds of incentives in their benefit designs. We need to incentivise consumers to look not just at cost, but to also look at quality. A second thing we have to do we've talked about, and that is making sure the information that we put out there is meaningful and relevant and consistent. And then finally, I think providers have a huge role to play. They know better than anybody the kind of information that will be most useful to consumers and most central to quality improvement. They need to play a very central role in shaping the measures of quality. And then they also have to play a role in reinforcing consumers' use of that information. The relationship between patient and physician is one that is still very critical to the decision that consumers make, and so the role the physicians play in responding to this information will have a great deal of influence on how consumers use it.

Penny Daniels: Debra, thank you so much. Now, in just a few moments we're going to open up our discussion for questions from you, our listening audience out there. I want to tell you that there are two ways you can send in your questions. We encourage you to ask questions by telephone. If you're already listening by phone, all you have to do is press *1 to indicate that you have a question. If you're listening through your computer and want to call in with questions, dial 1-877-407-3039, then press *1. While asking your question on the air, please do not use a speakerphone to ask your question because we will get feedback, it'll be hard to hear you, and if you're listening to the audio through your computer, it's also important that you turn down your computer volume after you speak with the operator, because there's a significant time delay between the Web and the telephone audio and it could get confusing for you. If you would like to send in your question by E-mail, all you have to do is click on the button marked Q & A on the Event Window on your computer screen, type in your question and then click the send button. One important thing, if you'd rather not use your name when you call us or communicate with us, that's okay, but we would like to know what state you're from and the name of your department or organization. So, please provide those details regardless of the way in which you send your question.

Now when you're formulating your questions or queuing up on the phone lines, I'd like to say a few words about our sponsors today. The mission of AHRQ is to support and conduct health services research designed to improve the outcomes and quality of healthcare, reduce cost, address patient safety and medical errors, and broaden access to effective services. We would appreciate any feedback you have on this Web conference. At the end of today's broadcast a brief evaluation form will appear on your screen. Easy to follow instructions are included on how to fill it out. Please take the time to complete this form, and for those of you who have been listening by telephone only and not using your computer, we ask that you stay on the line after the Webcast because the operator will ask you to respond to the same evaluation questions that we're getting in print using your telephone keypad. Your comments on this audio conference will provide us with a valuable tool in planning future events that may better suit your needs. Alternatively, please E-mail your comments to AHRQ at Okay, it's time to take questions from our audience.

And our first question, the person is on the phone. Janise calling from Sarasota, Florida. How are you? Janise are you there? I don't think we have Janise. Sometimes we have-you know this is live, and sometimes we have problems. So now this next question is for Denise. It's an E-mail question. Denise has AHRQ, or does AHRQ plan to report the QIs by hospital on a national basis?

Dr. Remus: Actually no, we have no plans at the agency to report on quality at the hospital level. The Healthcare Cost and Utilization Project dataset that we have actually has restrictions on how AHRQ can access hospital level data. If you're familiar with our National Quality Reports, you'll recognize that the goal there is to provide a picture of healthcare at the national level, regional and state. But we really rely on stakeholders out there to apply and develop performance measures in a sense, or apply performance measures to hospitals within our program. There's a lot of uniqueness about the discharge data across states and across hospitals that individuals who are setting up these programs have to have familiarity with. So I can clearly say that AHRQ has no plans now or in the near future to report quality at the hospital level.

Penny Daniels: Okay, thanks Denise. Sounds like Denise, Janise that is, is on the phone now from Sarasota, Florida. Janise, hi, what's your question? Do we have Janise. Oh, it's Janice, maybe that's why-is it Janice and is she on the phone. Janice speak to me please. We're having a little problem with Janice's audio, not with Janice herself. Can we have another question? Thank you. Here is a question from David Edwards. The question is, 'Risk-adjustment is an imperfect and developing science that many seem to believe is more accurate than is the case. Interestingly there isn't public reporting of risk-adjustment. So are there plans for us to AHRQ to discuss the limitations in the current risk-adjustment tools and to help groups understand how to interpret the data in light of this. Is there a plan to update in a public manner when improvements do occur in the risk-adjustment tools? Who wants to take that question?

Dr. Foxhall: Denise has got that question. It's a very good one, that's a very good one.

Dr. Remus: Actually there's two components to that that AHRQ is involved in. I think the first is we do know that risk-adjustment is an inaccurate science, and the reality is, its inaccuracies are based on the data and information available. So we're tackling that question from two different pieces. One is that we have a project that was just funded where we're going to be working with the state of Pennsylvania and MediQual to look at the clinical data that's available within their unique state dataset to identify whether certain clinical data elements when added to administrative data can in fact help not only with performance measurement, but with risk-adjustment as well. So one of the streams we're taking to work on some of the challenges of risk-adjustment is by enhancing the data that we have available to be used for risk-adjustment. The other component is that we are in fact, undertaking an evaluation in the coming year under the Support for Quality Indicator contract, to look at the risk-adjustment methodologies that we apply to administrative data, to look at more closely our use of the APR-DRG system, to look at more closely the diagnoses and procedures that we have available and look to enhance that risk-adjustment process that we use in the QIs. And we hope to have the outcome of that risk-adjustment evaluation completed in about 12-18 months.

Penny Daniels: On the same subject, briefly, I'd like to hear from Andy and/or Lewis, how did you handle risk-adjustment and is this information available on the Web? Maybe we'll go to Dr. Foxhall because he is not in our studio with us and we'll give him a chance to speak. Lewis?

Dr. Foxhall: Yes. This is really one of the most challenging areas that we have. The main thing I think we tried to concentrate on was finding a methodology that was open and understandable by the reporting entities so that they could have some level of confidence that they were being treated fairly. The first thing that happens when data is reported publicly is that if one is not at or above average, then it's very easy to attack the data. So having a process by which the differences in patient populations can be accounted for is very, very important. It's a challenging thing because many of the conditions or co-morbidities that are required to be reported and go into the risk-adjustment may not be things that are commonly noted on the billing form. So it takes some education and some understanding on the part of the hospitals to account for those things and to make it work right. So, it's still not perfect but I think it's at least a good effort at trying to take into account those differences.

Andrew Webber: Penny, just very briefly on both the questions, I think Debra and I would agree that we are committed in moving public reporting forward to give the explanations and the necessary qualifiers, and to explain risk-adjustment as an element of any public reporting. And I think that's an important complement to making the information transparent and out there in the public. The second thing I actually-because the study that I showed was a study by one of our members, I need to refer folks to the Alliance organization in Madison, Wisconsin, and maybe if my staff at NBCH is listening to this they could E-mail in with the Web site for the Alliance organization in Madison, and maybe we can announce it then.

Penny Daniels: Absolutely.

Andrew Webber: And John Bott, an individual at Madison Wisconsin is sort of the content expert for the Quality Counts Program and is the individual that someone might want to talk to.

Penny Daniels: Okay, thank you. Now Janice, from Baptist Health Florida is on the phone right now. Janice.

Janice: Yes.

Penny Daniels: Please ask your question. I apologize for mispronouncing your name.

Janice: I didn't know it was because I'm in Miami and you were saying Sarasota. Sorry about that.

Penny Daniels: I'm getting misinformation.

Janice: No problem. I'm definitely very excited about the way things are moving and that we're all working together nationally to measure quality, and I think it's a wonderful thing. But as you mentioned a moment ago, the difficulty of risk-adjustment is one of our big challenges. And another big challenge that I wondered if you could address is the issue of coding differences from facility to facility. And the example that I like to give also relates to clinical care practices differences. That would be DVT rates. DVT and pulmonary emboli, it's one of the patient safety indicators. And we compared some hospitals and saw that one of the hospitals had a higher rate of that complication than other hospitals. And then when we looked at the data closer, and we looked at the utilization of resources we found that the hospital with the higher rate was screening and doing Doppler studies to identify DVT, and therefore were maybe picking up some sub-clinical DVT. On that measure that hospital would not want that to be publicly reported because it's not necessarily a fair reflection of the work that they do. In their case they're actually being more diligent than many hospitals. So that being said, the two things I'm interested in is coding difference and how that can be addressed nationally to create more consistency, and secondly, the testing of the Patient Safety Indicators with real hospital data before they go to public reporting to make sure that they're fairly used.

Penny Daniels: Okay, complicated two-part question. Andy?

Dr. Remus: Yes.

Penny Daniels: Oh, Denise do you want to start with that?

Dr. Remus: Yes, I think I'll tackle that one. Thank you very much for calling in, and you raised some very good points. The first one about coding differences and how to address nationally is something that, I think those of us that work in the world of performance measurement understand the strengths and limitations of administrative data. And while there's not enough time to address that in detail, what I will let you know is that certainly we are working both with an HCUP program and other activities in AHRQ, to look at some of these issues of coding differences, not only across hospitals but across states. And looking at standardized audits, again, how to improve the quality of the data, especially with the transitions with HIPAA standards, which many of you providers and purchasers are aware of. And what I would encourage you to do is stay in touch with the HCUP program and the Quality Indicator program, as we move through some of these efforts. One of the things that you will see, that Andy briefly mentioned, is that we are developing a QI curriculum. That curriculum training session with be at the NADO (ph) meeting in December, and at the NBCH meeting in November. Additional information is on the QI Web site which the ULP address for that is on one of the slides that you'll see in a minute. But please stay in touch with us on that, because included in that training will be information on how to assess and evaluate the quality of administrative data. As far as the testing, there's no question that research and development is ongoing with these indicators. We have several projects with ongoing and collaboratives with UHC, with the VA, with DOD and other entities to look at the validity of the administrative data compared to the clinical data, and you're right, there's no question that anyone who's doing a performance improvement project or measurement effort needs to have an idea of the understanding of the data that they're obtaining, and it's quality before they move forward with using that information in any reporting program, whether it be for quality improvement, public reporting or pay-for-performance. So one of the fist steps you need to get through is having an understanding of the data that's available to you, and evaluating that carefully before you move forward into the next steps.

Penny Daniels: Thanks, Denise. Now several people have E-mailed in the same question so we're going to talk about that now. Why not reward other people such as nurses or healthcare systems instead of just doctors or providers?

Debra Ness: I'll take a stab at that Penny. As a consumer I would like to see us evaluating all types of providers and at all levels within the system. I think all of that is information that's important to consumers when they're making decisions. So I hope that's a goal that we're striving towards. I think we have to get there, and we're clearly starting in places where we have the capacity to measure, and hopefully we're starting in some of the highest impact areas. But really, the more ground we can cover the better.

Penny Daniels: Okay, Andy-you have something to add.

Andrew Webber: Well I just wanted to agree with that, and I think there is some positive movement on other fronts. NCQA has certainly led the effort with their HEDIS measurement set for the comparative performance of healthcare plan level. We actually at NBCH have a standardized request for information that provides additional information to employers that they can use. I think the whole arena of physician level evaluations, those at the medical group and individual doctor level, is very important. We've got a member in California, Pacific Business Group on Health working with six different healthcare plans on a common set of performance measures at the medical group level that is then leading to a pay-for-performance program, where up to $100 million is being moved around the system based on performance. We're working with a program called Bridges to Excellence, which is a program to evaluate performance at the individual physician level in the domains of diabetes, cardiovascular care, and whether or not physicians are making needed investments in health information technology. That program, based on a physician recognition program, that again NCQA developed, is being implemented in four communities, and we're hoping through out national distribution network of local coalitions to try to put some more energy in that program and get it expanded in other communities.

Dr. Remus: If I could just jump in. Also the National Quality Forum recently endorsed a set of nursing sensitive performance measures as well, so I think we are moving in that direction.

Penny Daniels: Okay, thank you all. This question is from Chris Wilson from the Advisory Board Company in Washington, DC, and it's for both Lewis and Andy. 'How have hospitals in Texas and Wisconsin adjusted their infrastructure and resources to adapt to the reporting of Quality Indicators?' Lewis, do you want to take that one first?

Dr. Foxhall: Sure. I really just have some anecdotal information on this, but there is one large hospital group, the Dallas Fort Worth Hospital Council that has done quite a bit of work with using the indicators. And they take the dataset from the Council and distribute it to their 60 hospitals, and they provide them feedback on the inpatient indicators, the safety indicators, and the prevention quality indicators, and help work with them on their Quality Improvement initiatives. They've been also trying to get additional funding. I believe they just got a grant in collaboration with Texas A & M University-Baylor, and another regional hospital area that helps bring this information out to smaller community hospitals through a Web portal. So I think there is definitely movement on this front and our own facility made some significant changes when we saw one of our indicators was a bit out of line. We're a cancer-only facility, so some of that of course was thought to be due to risk-adjustment differences, but nonetheless, that launched a big effort and we saw a very dramatic drop in the pneumonia rate, which was the one that we were looking at. So these things are happening and I think if, you know, pay-for-performance is important, then I think the public reporting also servers as a stimulus and certainly gets people's attention to begin to look at indicators and take action where it's needed.

Andrew Webber: Penny one of the risks of talking about programs of NBCH members is that I get asked very detailed questions, but I'm happy now that we're about to put up on the Web site the name of John Bott and his E-mail address. He is the individual at the Alliance that was really sort of the architect of the program. And if anyone has detailed questions, and needs some more information, again, I just had a chance to talk very briefly about the program, it's a very interesting project. John Bott's E-mail is coming up. Is it Penny?

Penny Daniels: It's flowing across the top of your screens. So for our listeners, right across the top of your screen you will see the Web site that Andy is referring to. It's Okay, there you go.

Andrew Webber: Thank you.

Return to Web Conference
Proceed to Next Section

The information on this page is archived and provided for reference purposes only.

AHRQ Advancing Excellence in Health Care