Skip Navigation Archive: U.S. Department of Health and Human Services
Archive: Agency for Healthcare Research Quality
Archive print banner

This information is for reference purposes only. It was current when produced and may now be outdated. Archive material is no longer maintained, and some links may not work. Persons with disabilities having difficulty accessing this information should contact us at: Let us know the nature of the problem, the Web address of what you want, and your contact information.

Please go to for current information.

Transcript of Web-assisted Audioconference

Session 3: Using Data To Tell The Safety Net Story

This Web-Assisted Audioconference consisted of three sessions broadcast via the World Wide Web and telephone September 23, 24, and 25, 2003. It was designed to inform State and community officials about and teach them to use Data Books and tools for monitoring the health care safety net. This initiative consists of a broad range of local area measures related to safety net providers and the populations they serve. The User Liaison Program (ULP) of the Agency for Healthcare Research and Quality (AHRQ) developed and sponsored the program.

Cindy DiBiasi: Good afternoon. Welcome to Using Data To Tell The Safety Net Story. This is the final event in a series of Web-assisted audio conferences on monitoring the healthcare safety net. These events are designed for state and local health officials. The series is co-sponsored by the U.S. Department of Health and Human Services Agency for the Healthcare Research and Quality or AHRQ, and the Health Resources and Administration or HRSA. My name is Cindy DiBiasi and I will be your moderator for today's session.

In 2000, the Institute of Medicine released a report describing the healthcare safety net as "in tact, but endangered." The safety net, as you know, is the nation's system of providing healthcare to low income and other vulnerable populations. In particular, the report emphasizes the precarious financial situation of many institutions that provide care to Medicaid, uninsured and other vulnerable patients. It also examines the changing financial, economic and social environment in which these institutions operate. It looks at the highly-localized patchwork structure of the safety net.

One of the five key recommendations in the report focused on the need for data systems and measures to assess the performance of the safety net and health outcomes of vulnerable populations. In response, AHRQ and HRSA are leading a joint safety net monitoring initiative. This initiative involves a three-part strategy focusing on both safety net providers and the populations they serve. As a result, they resolved to create two data books that describe baseline information on a wide variety of local safety nets, develop a tool kit for state and local policymakers, planners and analysts to assist them in monitoring the status of their local safety nets, and identify the data elements that would be needed to successfully monitor the capacity and performance of local safety nets. All of the information related to the AHRQ and HRSA initiatives is available on AHRQ's Website at

As we talk about the safety net, it is important to make sure there is a common understanding regarding what it encompasses. The healthcare safety net consists, as we said, of a wide variety of providers delivering care to low income and other vulnerable populations. These include the uninsured and those covered by Medicaid. Many of these providers have either a legal mandate or an explicit policy to provide services regardless of a patient's ability to pay. Major safety net providers include public hospitals and community health centers as well as teaching and community hospitals, private physicians and other providers to deliver a substantial amount of care to these populations.

In the past two Web-assisted audio conferences, we have examined the new data books and data collection strategies outlined in the safety net tool kit. Like the data book the tool kit is designed to help policy analysts and planners at the state and local levels assess the performance and needs of their local safety nets. Chapters included in this book are written by experts in the field covering a wide variety of topics.

Today we will examine three more chapters examining how to use data to tell the safety net story. These reports are included in the forthcoming book entitled Monitoring the Healthcare Safety Net, Book III, Tools for Monitoring the Healthcare Safety Net. The book will be available later this fall and will tell our listeners how to get this new book at the end of today's discussion. But let me begin by introducing today's panelists. In the studio with me I have Pete Bailey, chief of health and demographics for the South Carolina Budget and Control Board. Andrew Bazemore, assistant professor in the Department of Family Medicine at the University of Cincinnati, Christine Shannon, administrator in the Office of Health Planning and Medicaid for the New Hampshire Department of Health and Human Services. Joining us remotely is John Billings, director for the Center for Health and Public Services Research at New York University's Wagner School of Public Service. Also here with us here in the studio is Robin Weinick. Robin is the senior research scientist and senior advisor on safety nets and low-income populations at AHRQ. As the lead on AHRQ's Safety Net Monitoring Initiatives, she will stay with us today and join us during the question and answer session. Welcome everyone.

Before we begin our discussion, I would like to tell the audience a bit about the format of this Web-assisted audio conference. First we will talk with our three panelists and then open up the lines to take your questions. We will give instructions on how to send your questions to us later on in the program.

In the meantime, if you experience any Web-related technical difficulty during this event, please click the "help" function in your window to troubleshoot your Web connection. If it appears the slides are not advancing, you may need to restart your browser and log on again. If you are on the phone, dial "*0" to be connected to technical assistance. Also, if you have difficulty with the audio stream or if you experience an uncomfortable lag time between the streamed audio and your slide presentation, we encourage you to access the audio via your phone. The number is 1-888-469-5316. This is the same number to call to ask questions when we get to the question and answer portion of the program. Because of the technical nature of today's call, we highly recommend that you download slides from today's event. You can do that by logging on to

Well now I think we are ready to tackle today's topic, Using Data to Tell the Safety Net Story. Let's begin with John Billings, director for the Center of Health and Public Service Research at New York University's Wagner School of Public Service. John, was not only a primary collaborator in developing the two data books; he also wrote the safety net chapter entitled Using Administrative Data to Monitor, Access, Identify Disparities and Assess Performance of the Safety Net. John, exactly what are administrative data? Can you give us an example of that?

John Billings: Sure. Administrative data are computerized records. They are usually gathered for some administrative purpose, hence the clever title "administrative data." They are used generally for bill paying, reimbursement, and record keeping. The interesting thing is, they typically have lots of information about individuals: demographics, age, sex, race, and ethnicity. Then a lot about what services have been utilized, how the money flows, how the money is being spent. Then other events like births or deaths. Some examples are, as I say, birth and death records, which are very interesting for looking at things like prenatal care. Then hospital discharge records that record admissions to hospitals and discharge from hospitals. Increasingly used are our emergency department records, which talk about or have evidence and information about visits in emergency departments. Then there is a long history of using Medicare and Medicaid claims files. These are the records that record the payments from the Medicare and Medicaid program. They also tell a lot about where the money goes and how services are utilized.

Cindy DiBiasi: What are some of the advantages and disadvantages of using this kind of data?

John Billings: Well, the main advantage is they are already there so you don't have to go out and collect them. Another important advantage is they have already been computerized so you don't have to be the one who has to put them on the computer and be responsible for all the errors. Now they can be also relatively inexpensive to get because they have already been gathered and it is just electronic data. Now that is not always true. Sometimes some places charge more than others. But they are also generally inexpensive to analyze because again they have already been computerized. They can often tell you something very interesting about what is going on in your community with respect to the safety net, where people are getting services, what kind of service they are getting and a little bit of something about the kind of barriers they may be experiencing.

Do you want me to talk a little bit about the disadvantages, of which there are some?

Cindy DiBiasi: Yes, yes, please.

John Billings: A disadvantage, caution is required in using administrative data. Remember, they have been gathered for something else. So they could be dirty. Some elements are often much better than others. A good test is usually if someone is going to go to jail if the data is bad, which is not most of the data elements, but sometimes that is true. For example, on hospital discharge data, the diagnostic fields are very important. That is the way people get paid and there are penalties for fraud if those fields are filled out inaccurately. On the other hand, the fields about race, ethnicity or the source of admission, other things that aren't so important to payment are less and less likely to be accurate.

The second limit that people need to keep in mind when they use this kind of data is they seldom tell the whole story. You often raise as many new questions as you can answer the old questions. That is often a very useful process, but you would need to go into it understanding that you are not going to get the whole story from administrative data, but they often point you in the right direction.

Another problem can be, especially with something like emergency room data where there are only a few states that have that data available and not everyone is willing to share, which is required. If you are going to look at emergency room use in a community to understand how the safety net is working, you really pretty much need to go to all the hospitals in the community to cooperate. Not everyone is always willing to cooperate. They sometimes see it as private information.

Finally, even where you are getting it from the state, for a relatively low cost, you are going to be dealing with people who have other jobs to do and helping you get the data and getting it to you in a timely way isn't always their first priority and so you need to recognize that there could be trouble in getting it. But once you get it, you are halfway there.

Cindy DiBiasi: Now how would you measure something like birth outcomes, for example?

John Billings: Well, birth records have been used by lots of people for many years to look at two or three things. One of the most important things, obviously, that has been looked at over time is infant mortality. Thank goodness that is becoming a rarer and rarer event, so people are looking more at things like did the mother get timely prenatal care? So the rates of later prenatal care. There is also interest in looking at things like low birth weight, which might tell you something about the kind of care the person got or some of the problems they may have in their lives. Also pre-term birth, which tells a similar story about access to care and the social and economic circumstances of the parents.

Now we have tried to look at some of this data in many different ways. We find it very useful to get to the lowest geographic area possible when looking at things like the rate of prenatal care. Then there are lots of ways to display it graphically, which are very intuitive to policymakers. One of the impressive things about using this kind of data is that policymakers get it. So if you show a chart that says, all right, on the Y-axis, is the rate of late or no prenatal care; on the x-axis is how poor the neighborhood is and each dot represents or each square represents a zip code in a city. It is very easy for a policymaker to see, for example, that low-income areas tend to have much higher rates of late or no prenatal care than high-income areas. They can see it in a glance and it conveys a complicated story very easily.

Then we often in many circumstances try to map the data so you could take the zip code-level data or county-level data and you can then map it to display in the community, which again helps policymakers identify the areas in the community that have the most severe problems. For example, in New York City we found that some of the neighborhoods where there were relatively small African-American populations, were some of the neighborhoods where the African-American mothers were getting the worst health outcomes, the worst prenatal care rates. So by mapping them out, you can really see a trend that you might otherwise miss.

Cindy DiBiasi: What about hospital discharge data? What can we learn about utilization of services by analyzing that data?

John Billings: People have been looking at that for years. With respect to the safety net, what is often used is something called Ambulatory Care-Sensitive Conditions or Preventable Hospitalization. These are things where timely and effective care outside of the hospital in the ambulatory care setting and help prevent the need for hospitalization. Chronic conditions like asthma or diabetes or congestive heart disease, acute conditions like ear, nose and throat infections, cellulitis, pneumonia and then totally preventable things where if you got for example, an immunization you won't get the illness. These are things where if you got timely and effective care, the rate of hospital admissions from the area or for the population should be much lower. So people have used these sorts of things to again, look within a community to try to understand which areas of town are having some of the worst health outcomes or have the biggest potential barriers to getting timely care and which populations are having similar problems. So the example, we have looked at Baltimore at the zip code-level and saw again much higher rates of these preventable hospitalizations among children in low-income areas compared to children in high-income areas.

Another interesting thing to do is to look at different populations separately. So if you look at children you may see one pattern. Then if you look at adults, typically you see a somewhat different pattern. That is not necessarily surprising and can help illustrate something important to policymakers and that is that we have invested a lot of money in improving access to children whether it is through the health centers sponsored by HRSA or through a stanchion of insurance company coverage through Medicaid and CHIP programs. What you see when you analyze that is disparities between rich and poor neighborhoods are generally much smaller for children than they are for adults. That can be an enormously useful thing for policymakers because we seldom have good news stories to tell on health and there is a good news story. We invested money and we saw some good health outcomes. But also helps people understand where the problems might be and where they need to focus their intervention.

Cindy DiBiasi: Now I know you also looked at ACS admissions in Atlanta. What did you find there?

John Billings: Well, as usual, we found big differences depending on what part of the community you lived in. We first did the work at the county level, which is something that is very appealing to people. We mapped that out and when we looked at Atlanta at the county level, looked at the MSA of Atlanta at the county level, you really didn't see an enormous difference among counties. It really wasn't providing much useful information. But if you get a little better resolution by going down to the zip code-level, which in most hospital discharge databases you can, you suddenly see some counties that on average look pretty good. Some parts of the county looked much worse than other parts of the county. Those are typically areas that have a lot of vulnerable populations, either lots of low-income patients or minority populations, immigrant populations. Again, by mapping it out it can be very useful to policymakers to both number one illustrate that there are disparities within a county or within a geographic area, but more importantly to help them target where you want to go and do something about it.

Cindy DiBiasi: What did you see when you looked at this in New York?

John Billings: In New York, we have been looking at this for almost 15 years, I am ashamed to admit. You find lots of interesting things. For example, there is obviously a very strong correlation between income and preventable hospitalization rates. That is the higher the hospitalization rate, generally the lower the income of the neighborhood. But that is not always the case. Some neighborhoods that are equally poor have incredibly different admission rates. For example, in one part of New York the admission rate for a low-income neighborhood is about three times that of another neighborhood in New York. That helps you understand that this isn't just a poverty issue; it has something to do that is much more complex with the nature of the population being served, but also how well the healthcare delivery system is performing in that area. So by illustrating that you can see two zip codes that have similar demographic characteristics, but have dramatically different hospitalization rates, it can help you start thinking about how to make an intervention.

Cindy DiBiasi: John, tell us about the algorithm that you developed for using emergency data.

John Billings: Well, there is increasing interest in what is going on in the emergency room. Hospitals have to take patients who show up in the emergency room by federal law. Therefore, the emergency room is being increasingly recognized to be the safety net for the safety net. So looking at the pattern of utilization of patients who come into the emergency room, we thought it might be a useful way of understanding something again about the nature of the barriers to care and the community and where those barriers might be. So we together with others, developed an algorithm that tries to classify emergency room use into four categories: non-emergent care. These are things where you don't need to be seen today. A sore throat is pretty typical of that. Emergent meaning you need to be seen today, but it could be primary care treatable meaning you don't need to be in the emergency room. That might be a child, an infant with a 102-degree fever. Well, it would be appropriate to see the physician today, but you don't need to rush to the emergency room. You can go to your own doctor. Then there is a whole series of things where you are in the right place if you go to the emergency room. If you are having chest pain, please go to the emergency room. But there are other things where we want you to go to the emergency room that if we had seen you earlier in the episode maybe we could have prevented it from becoming so acute. You needed to go. In a classic example, that might be a diabetes chronic-acute attack, that raises your blood sugar level and suddenly you need attention. If we treated (unclear) you had been able to go to the doctor earlier in the week, you won't show up with ketoacidosis in the emergency room on Thursday. So again, it has been a tool that we then could look at a zip code level and look at, compare one neighborhood to another. We did that in Baltimore and in several other cities where we could again see a very strong association between emergency room use for things that are preventable and avoidable and the income of the neighborhood. Again, the poor in the neighborhood had much higher rates of emergency room visit rates for things that are preventable or avoidable.

We have also again mapped it out periodically. For example, in Austin, Texas for one of the CAP programs that HRSA sponsored, they wanted to look within their community to try to identify where some of the biggest barriers were and where people were using emergency rooms that maybe they could prevent that and so they were able to apply this algorithm and map it out and show which parts of town had the highest rate.

Cindy DiBiasi: What is that line going through Austin?

John Billings: That lovely red checkered line? Well, when you are doing maps, even for people who have lived there their entire life they get disoriented. So we found it very useful to put geographic markers on the map to help orient people. That is the highway that goes through the middle of Austin so people know which side of the highway they live on. They know which county they live in and so, they may not know the boundaries of their zip code, but once you put some identifiers on there they can help orient themselves. So that is useful to policymakers and to the press and other people who might be looking at this sort of stuff.

Cindy DiBiasi: So based on your experience working with this kind of data, any words of caution that you would like to share with us?

John Billings: Yes. This is something where you have got to be very cautious as you are using it. As I said earlier, the date could be dirty and it is important that you do some tests to make sure you are looking at what you really think you are looking at. Then if a number is way high or way low, it is usually something wrong with the data. Now that is not always the case because disparities in health outcomes can be astoundingly enormous. But my first reaction is always I did something wrong or there is something wrong with the data and so it is important to go check it out before you announce to the world that there is a problem or not a problem.

Then as I said earlier, you can't expect often from this data is the final answer. What you can expect is the targeting of the next set of questions or focusing of future work to try to figure out what to do next.

The final thing I would say, a final caution, is to avoid easy explanations. When we started looking at these preventable hospitalizations, the first explanation was well, there are not enough doctors. We need to get more doctors in the community. In New York we invested an enormous amount of money in expanding primary care capacity. Whereas that was pretty important, there were lots of other things that might have explained why people were not getting timely and effective care. They didn't know when to come, they didn't know where to go and they didn't like where they were going. So it wasn't necessarily just that there wasn't a doctor there. It can be a much more complex issue about the social dynamics of the family, but also the performance of the safety net in the community.

Cindy DiBiasi: John, thanks. We will come back to you during the question and answer period so hang on with us.

I will turn now to Andrew Bazemore, the assistant professor in the Department of Family Medicine at the University of Cincinnati. Andrew is the co-author of the safety net tool kit chapter entitled Mapping Tools for Monitoring the Safety Net. Andrew, let me start with the Geographic Information System, the GIS. What is that?

Andrew Bazemore: Cindy, in the broadest sense, a geographic information system is a tool that is capable of linking together any data as long as it has a geographic location or address attached together with map features. In recent years, what this has really meant is large software programs, particularly as we have seen computer data handling and graphics ability increase. The software programs can bring together large data sets, such as the ones John is talking about, that then allow us to collect, to retrieve at will, to transform and display spatial data from the real world. So really what GIS allows us to do is to analyze and transform complex data from many, many sources together into maps that as John as already demonstrated, will illustrate problems effortlessly for experts and non-experts alike.

Cindy DiBiasi: You were mentioning that it integrates any existing data that has a location or an address. What sort of data can be used for mapping in healthcare?

Andrew Bazemore: Well, John has given us a great lead-in here. Basically any number of administrative and claims data can be put together. For example, in the safety net, the community health centers are required to collect for the federal government, data from every single patient visit into a uniform data set. So we can take their UDS data, their patient billing records. We can put that together with population data such as from the U.S. Census. We can add in if we want insurance or claims data. We can put, as John mentioned, highway data, transportation, roads or bus lines. We could put on top of that city planning data such as from the waters and utilities division. If we wanted to, we could overlay satellite or mapping data and finally as mentioned, there are many, many sources of data from local, regional and state health departments.

Cindy DiBiasi: And how is that going to be helpful and how can safety net clinics use this technology?

Andrew Bazemore: Well, I would say the possibilities are fairly endless. I will give you a few examples. For one, if we took the community health centers uniform data sets data and we put it together for them, we can actually show exactly what service areas their clinic serves, by individual clinic. On top of that, if we overlay population data from the census, we can allow these same clinics to look at their market penetration rates, all the way down to areas that are only a few blocks wide. Now, as you probably know, the community health center networks are required to attempt to serve at least, medically-underserved areas as defined by the federal government. When we overlaid the maps of these medically-underserved areas, we could allow these community health centers to see whether or not they are succeeding in this process. Particularly, we then breakdown the medically-underserved area to look for target areas or regions where they can better serve either the entire population or a particular at-risk subset, say by age, race or gender.

Now all of these functions really come together to allow a clinic to better provide community-oriented primary care. By that I mean instead of just focusing on the patients walking into the clinic doors, they can really look at the needs of the community and the communities that these patients represent.

Ultimately, this would be best done if we were actually able to put this on the Web. If were able to let the users, the clinicians and the leaders to actually go with their questions that come up on a day-to-day basis and withdraw their data so long as it was securely stored at a Web-based site.

Cindy DiBiasi: Let's look at some of the areas that you have mapped.

Andrew Bazemore: Moving on, you will see in this first map, Boone County, Missouri. The black boundary lines you see actually represent U.S. Census tracts. Now the dead center of Boone County is Columbia, a medium-sized city and also the county seat. The map on the left shows you census tracts, those being a small division done by the Census to break down population groups. These were the tracts, in white, that were federally designated as medically-underserved areas for that county. As we just said, these were the areas that the community health centers, of which there are two in Boone County, are supposed to target when they apply for their grant funding. However, when we took the actual clinic data and we turned it into maps, as you see on the right, the new white areas are their actual service area. What you will see is the two don't exactly match. We could take this one of two ways, but Boone County's community health centers wisely looked at this discrepancy and didn't see service failure. They saw opportunity. So they were able to return and say we need to actually expand our services. We need to go out into the county and provide more service than we did before.

Now another example in the next slide is a clinician in Baltimore, I was working with the largest network of community health centers called Baltimore Medical Systems. What we did was take their four largest centers and try to map their service areas, which you see in green, blue, red and yellow here. Then on top of that, you are going to see some brown and some pink-shaded areas. These are the tracts where the health service areas actually overlap. Now this was particularly helpful to the administrators of the Baltimore Medical Systems clinic and recognizing for the first time that their own clinics actually had overlapping service areas. We actually had two clinicians pull us aside and say, "I'm glad you took some heat off of us. We were hearing that our inability to fill our clinic schedule was due to loss to competition." Well, their competition it turns out was probably within their own network.

We look at the next slide. Similar to Boone County, Missouri, this map is going to show you the medically-underserved area census tracts, in pink, which were supposed to be served by one of the Baltimore Medical Systems clinics. What you see with all these concentric overlay circles are quarter-mile intervals moving out from this clinic. The big, blue circles are at one mile and two miles respectively out from the clinic. We were asked early on in the mapping process, it is kind of a sample question, could we evaluate a pending decision by the clinic to move itself approximately two miles outward from its current location? They said we are well within what the federal government has told us that this is our medically-underserved area if we go to the upper right hand corner of this map. We said absolutely. But if we then look at the next slide, the bottom left hand corner of this new map shows us the actually placement locations from the previous year for that clinic. The clinic told us they were particularly concerned about pediatric patients and African-American patients. So we actually took the target population and mapped that. What you will see is an enormous cluster of these patients sitting in the bottom left hand corner, not the upper right hand corner of the map. Essentially, our data along with another of the maps that I don't have here to show you allowed us to help them change their minds about moving to save a few dollars on leasing. We feel that through this mapping really gave them a large benefit.

Cindy DiBiasi: Are there any obstacles to using and interpreting these maps that make it more difficult for some of these community health centers to replicate what you have been doing?

Andrew Bazemore: Well John wisely said whenever you use data you have to be very careful in the way that you interpret it. For example, as we have found in moving from Boone County to Baltimore, it is very challenging to find the ideal service area or the ideal penetration rate for a community health center. There are a myriad of safety net providers from the emergency rooms to clinics to hospital outpatient settings that all would lay claim to a certain group of targeted population. It is very difficult again to say this is the group that we should go after. However, again, ideally we bring together all sorts of data from the safety net and merge it in maps. Theoretically, we could actually map an entire urban region and show again the gaps and places of over-utilization with an entire urban area such as Baltimore.

The second big gap, of course, or the second big obstacle is the cost and the technical expertise required to form this level of mapping at the individual clinic level. Again, this is where Web-based mapping would come in and allow us not to need information technology expertise or expensive software for each clinic but actually take advantage of the economies of scale that a large, large group of clinics such the CHC's or other safety net providers can give us.

Proceed to Next Section

The information on this page is archived and provided for reference purposes only.

AHRQ Advancing Excellence in Health Care