Skip Navigation U.S. Department of Health and Human Services www.hhs.gov
Agency for Healthcare Research Quality www.ahrq.gov
Archive print banner
Monitoring the Health Care Safety Net

This information is for reference purposes only. It was current when produced and may now be outdated. Archive material is no longer maintained, and some links may not work. Persons with disabilities having difficulty accessing this information should contact us at: https://info.ahrq.gov. Let us know the nature of the problem, the Web address of what you want, and your contact information.

Please go to www.ahrq.gov for current information.

Developing Data-Driven Capabilities to Support Policymaking

Select for print version (PDF file, 395 KB). PDF Help.

By Robin M. Weinick, Ph.D., Agency for Healthcare Research and Quality; Peter W. Shin, M.P.H., Ph.D., George Washington University


Health care organizations are now emphasizing evidence-based medicine, which involves using research findings on the effectiveness of various practices to help make treatment decisions for patients. A parallel practice that is receiving increased attention is using data and the findings from data analysis to inform the policymaking process. The data-driven policy framework presented here involves an explicit statement of priorities and policy questions to be answered by new and existing data and provides general guidance for using data to support the process of developing policy options for the health care safety net.


Contents

The Safety Net Monitoring Initiative
A Process for Developing Data-Driven Capabilities to Support Policy Decisions
   Who Should Be at the Table for This Process?
   Definitions and Priorities
   Data
   Assessment
   Action
   Evaluating the Policy
Using Data for Developing Policy Options
References

The Safety Net Monitoring Initiative

In September 2003, the Agency for Healthcare Research and Quality (AHRQ) published two data books in cooperation with the Health Resources and Services Administration (HRSA). These books are entitled:

These books provide 118 measures to help policymakers, planners, and analysts monitor the safety net in 90 metropolitan areas and all 1,818 metropolitan and nonmetropolitan counties in 30 States and the District of Columbia. In conjunction with a third volume, Monitoring the Health Care Safety Net—Book III: Tools for Monitoring the Health Care Safety Net, these books form the core of the joint AHRQ-HRSA Safety Net Monitoring Initiative.1-3

To further assist State policymakers in using the information in these books, HRSA has funded a grant through the National Governor's Association Center for Best Practices entitled "Enhancing the Safety Net Through Data-Driven Policy." This intensive technical assistance project is designed to help policymakers in four States develop a series of data-driven recommendations to enhance the strength, structure, and stability of their health care safety nets. As part of the project, interdisciplinary State teams are using the two data books to access new information about their safety net systems, and are using these data tools as the basis for crafting their own policy initiatives to strengthen and sustain the health care safety net. (For more information, go to www.nga.org.)

Site visits to the four States (Arizona, Florida, Oregon, and Virginia) selected for this project were conducted in January and February 2003. This data-driven policy framework focuses on the process that evolved from those visits by which States can begin to develop their capacity for formulating data-driven policy concerning the provision, financing, and monitoring of the safety net.

For more information on these data books and the Safety Net Monitoring Initiative, go to http://www.ahrq.gov/data/safety net to download the books or request free copies, or call (800) 358-9295 and request publication numbers 03-0025/03-0026.

Return to Contents

A Process for Developing Data-Driven Capabilities to Support Policy Decisions

Figure 1 (23 KB) displays a four-stage process for developing data-driven capabilities to support policy decisionmaking.

Who Should Be at the Table for This Process?

Ideally, this process should involve all the major stakeholders with an interest in the safety net. This includes State agencies that are involved in providing policy direction, program management, and data collection. In addition, including representatives of associations of community health centers, free clinics, and hospitals as well as other providers involved in the safety net would be beneficial as well. Moreover, representatives from county and city health departments, special State and local programs, and organizations that collect health data add tremendous depth of perspective.

To help facilitate discussion, the agency or official responsible for convening the stakeholder organizations also should define the focus and charge of the safety net workgroup. For example, the workgroup may be charged with identifying ways to improve provider participation in the State Medicaid program or identifying data to help describe access problems in the safety net. Specifying the purpose of the work group serves to ensure that appropriate representatives (e.g., chief executive officers [CEOs] or data experts) are present to provide direction for developing viable policy options. In effect, this policy process relies on the workgroup involved to clarify questions about the safety net, share data and information, assess policy options, and describe policy recommendations sufficiently for effective implementation.

Return to Contents

Definitions and Priorities

Defining the Safety Net

All parties involved must begin by articulating a common definition of the safety net, which is usually defined in three primary ways:

  • By population.
  • By provider.
  • By funding stream.

These are not mutually exclusive, and States may choose to incorporate multiple aspects of each of these three dimensions in their definition.

Populations may include:

  • Low-income individuals or families (below 200 percent of the Federal Poverty Line).
  • Uninsured and/or underinsured individuals.
  • Beneficiaries of Medicaid or the State Children's Health Insurance Program.
  • Individuals with special health care needs.
  • Homeless individuals.

Providers may include:

  • Public hospitals.
  • Community health centers and other centers such as rural health centers, community mental health centers, and migrant health centers.
  • Free clinics.
  • Local health departments.
  • Emergency departments.
  • Community and teaching hospitals.
  • Medicaid managed care organizations.
  • Private physicians who provide charity care or Medicaid services.
  • School-based health centers.
  • Other providers who offer substantial services on a sliding-scale fee or at reduced prices for those who cannot otherwise afford them.

Funding streams may include:

  • Medicaid (Federal and State funds).
  • State Children's Health Insurance Program (Federal and State funds)
  • Medicare.
  • Disproportionate Share Hospital payments through Medicaid and Medicare.
  • Federal, State, and local funding for community health centers and other providers such as rural health centers, community mental health centers, and migrant health centers (including Federally Qualified Health Center funds, Section 330 Grants, Maternal and Child Health Services Block Grants, rural telemedicine funds, and cost-based reimbursement for Rural Health Clinics).
  • State and local programs that subsidize care for low-income persons or those with special health care needs, including State and county indigent care programs, public health programs, and programs to subsidize care for special populations.

Failing to begin with a shared definition of the safety net may cause confusion later in the process, as representatives from different organizations may be working under different assumptions.

Clarifying Current Concerns and Priorities

As the process begins, members of the workgroup should understand each other's respective concerns and priorities regarding the safety net. Developing a shared list of concerns and priorities can shape the overall direction of the project and help to more efficiently target data-based policy efforts. For example, the group may choose to focus its efforts on one or two priority areas, such as measuring capacity for primary care or mental health services for the uninsured, rather than beginning with a general review of all data available that may be relevant to the safety net.

Understanding Questions to be Answered

In the initial stages of formulating data-based policy options, begin by identifying what questions or policy problems the work group feels should be addressed. For example, if the group is convened to address the financial fragility or vulnerability of the safety net, they may inquire about the level of Medicaid revenues, uncompensated care pool payments, or local indigent program funds paid to safety net providers.

Return to Contents

Data

Assembling a Data Matrix

Figure 2 describes a data matrix that can help evaluate what data are available to support the policy development process. A data matrix serves as a reference tool for everyone working on the project, so that there is a clear understanding of what resources are available for answering key questions about the safety net. It should include all relevant data sources, regardless of where they are housed, and is likely to include data sets from multiple State and local agencies, Federal data collection efforts where relevant, and from non-governmental organizations that collect data (e.g., an association of community health centers).

For each data set that contributes to analyzing the priorities, the following questions should be addressed:

  • How important are the data? Describe the key questions the data source can address and the policy questions it can be used to answer.
  • What do the data represent? Describe the sample or universe from which the data are derived as well as the unit of analysis. Examples of the sample or universe can be all hospitals, persons covered by Medicaid, DSH funding sources, and so on. Examples of the unit of analysis can be one patient discharge record for a hospitalization, one claim covered by Medicaid, one DSH payment, and so on.
  • How helpful are the data? Include information on how frequently the data are collected, how accessible they are, and how much confidence experts have in the data.
  • How can the data be acquired? Include contact information.

Figure 3 provides a sample data matrix for the data sets used for the two data books from the Safety Net Monitoring Initiative.

Determining Available Measures

Without appropriate measures, determining baseline conditions or evaluating the impact of policy changes can be difficult. With that in mind, it is particularly helpful to understand what specific measures are available from existing data sources that can help track the effect of any actions.

Ideally, measures are:

  • Clearly defined.
  • Closely related to the types of changes being tracked.
  • Available from existing data.
  • Available on a regular basis to facilitate monitoring.

Examples of measure regarding the safety net include emergency department visits and cost of care per uninsured patient. It may be necessary to use a proxy measure, if a more direct, closely related measure is unavailable. For example, access to primary care can be measured by the volume and increase in emergency department visits for non-emergent care.

Identifying Need for New or Additional Data

Existing data resources may not be adequate to answer all the questions, provide a complete assessment of the current situation, or monitor anticipated changes. Although developing new data may not be a feasible option, it is always helpful to have a clear understanding of the limitations of currently available data and to know about additional or new data that would be helpful. It can be difficult to adequately manage or evaluate programs if data and measures are unavailable, so take the time to identify the limitations of current resources.

Inventorying Initiatives

In combination with data and measures, an inventory of current and past initiatives focused on the safety net can help provide context for developing new policy options.

Return to Contents

Assessment

Once definitions and priorities have been established and a data matrix has been constructed and shared among the relevant stakeholders, the data can be used to assess specific issues or problems to target policy options to the specific providers, populations, geographic areas, and so on, with the greatest need.

Analyzing Data

Figure 4 (13 KB) shows the Policy Analysis Framework originally developed by Larry Lewin, Jack Needleman, and David Helms for AHRQ's User Liaison Program.4 It provides general guidelines for using data to analyze the problem with appropriate questions and to assess the available options based on select criteria.

The matrix shown in Figure 2 can help identify data that can be used to answer these questions, as it is designed to answer commonly asked questions about the safety net with available data and to serve as a reference tool for other potential policy efforts.

Specific questions that are recommended for general analyses of the safety net are shown in the Safety Net Assessment Worksheet in the Appendix. Additional questions may be relevant, depending on the definitions and priorities that were selected at the beginning of the data-to-policy process.

Clarifying the Limitations of Current Knowledge

Understanding the limitations of current knowledge is crucial for using data wisely to support the policy development process. Be as explicit as possible in outlining the limits of what can be understood with current resources so as to be clear on the "blind spots" in the policy analysis process.

Disseminating Findings

It is often helpful to disseminate the findings from the assessment before or while developing policy options. Stakeholders may have a wide variety of uses for the information, and may also supply creative, viable policy options once they have had a chance to review the results.

Go to the chapter, "Presenting Information to Decisionmakers: A Guide for Policy Analysts," by Larry Lewin and Marion Ein Lewin, in Monitoring the Health Care Safety Net—Book III: Tools for Monitoring the Health Care Safety Net. To request free copies of this book, call (800) 358-9295 and request publication number 03-0027.

Return to Contents

Action

The Policy Analysis Framework shown in Figure 4 moves from an analysis of the problem, discussed above, to assessing and selecting policy options. Helpful questions to answer include are listed in Table 1.

Estimating the Impact of Current and Previous Initiatives and of Current Options

A description of the impact that current and previous initiatives have had can be helpful to understanding the likely effects of current options. It is particularly helpful to know about the unintended consequences of these initiatives to minimize such effects from any new policy actions. In addition, consider the evaluation methods for current policy options before they are implemented. Failing to do so may lead to options whose successes and failures cannot be readily measured.

Recommendations

Options and recommendations should be presented in clear terms, with graphic tools to help stakeholders understand the details involved.4 It is crucial to present the results of the data analyses that support the policy options being recommended. This helps garner support by providing a sound basis of evidence for the actions proposed.

Return to Contents

Evaluating the Policy

While this discussion began with establishing definitions and priorities, ideally the data-to-policy process forms a continual feedback loop. Once a policy option is chosen and implemented, repeated assessment can help measure its impact, establish its strengths and weaknesses, and provide information to support decisions regarding continuation of the initiative or a shift in strategic direction. Similarly, the effects of a policy initiative may change definitions and priorities for future work, beginning the cycle anew.

The Policy Analysis Framework also provides some helpful questions to ask when deciding among the available options as well as the implementation, support, and evaluation of a policy initiative. Some of these are outlined in Table 2.

Return to Contents

Using Data for Developing Policy Options

Figure 5 (11 KB) illustrates how both data and policy priorities influence policy options. Priorities provide a "top-down" perspective—from the big picture of a set of goals and areas of emphasis down to specific policy options. This approach involves a straightforward guideline for identifying relevant policy questions and goals first. The data analysis is regarded as the next step in which collection of new data may be required to support the information needs of the policy process. In contrast, a "bottom-up perspective" relies largely on existing data sources to identify or clarify areas of concern and to provide available evidence for action to contribute to the policy process.

In general, a top-down perspective provides greater flexibility in the scope of the safety net issues the workgroup can address, whereas a bottom-up perspective helps to identify current problems with existing data but may not adequately reflect policy priorities. Although the choice of perspectives in the policy development process may be influenced by stakeholder perceptions, political feasibility of policy options, timing, and available funding, data analysis remains crucial in the policymaking process.

As Figure 5 (11 KB) shows, data analysis can provide either supportive or contradictory evidence for the severity, prevalence, and causes of the problem and the development of reasonable policy options. In addition to the scope and seriousness of the problem, data analysis also can help maximize the desired outcomes of any policy initiative. In general, effective data-driven policy involves both perspectives, one focusing on the listing of priorities and policy questions to be answered by new and existing data and the other relying on a data matrix that can be used as a prepared reference for answering safety net questions as they arise.

Return to Contents

References

1. Billings J, Weinick RM. Monitoring the Health Care Safety Net—Book I: A Data Book for Metropolitan Areas. Rockville, MD: Agency for Healthcare Research and Quality; 2003. AHRQ Publication No. 03-0025.

2. Billings J, Weinick RM. Monitoring the Health Care Safety Net—Book II: A Data Book for States and Counties. Rockville, MD: Agency for Healthcare Research and Quality; 2003. AHRQ Publication No. 03-0026.

3. Billings J, Weinick RM. Monitoring the Health Care Safety Net—Book III: Tools for Monitoring the Health Care Safety Net. Rockville, MD: Agency for Healthcare Research and Quality; 2003. AHRQ Publication No. 03-0027.

4. Lewin L, Lewin ME. Presenting information to decisionmakers: a guide for policy analysts. In: Billings J and Weinick RM, editors. Monitoring the Health Care Safety Net—Book III: Tools for Monitoring the Health Care Safety Net. Rockville, MD: Agency for Healthcare Research and Quality; 2003 Dec. AHRQ Publication No. 03-0027.

Return to Contents

AHRQ Publication No. 04-0037
Current as of April 2004

 

The information on this page is archived and provided for reference purposes only.

 

AHRQ Advancing Excellence in Health Care