Informed Decisions Toolbox
Finding and Using Management Research Evidence
To promote evidence-based management in health care, the Agency for Healthcare Research and Quality (AHRQ) funded development of tools to help decisionmakers find and interpret evidence related to decisions.
The Toolbox was created to help bridge the gap between research evidence and organizational decisionmaking. It describes six steps for managers and policymakers to consider when gathering evidence to make a well-informed decision.
To help decisionmakers find and interpret evidence related to management decisions in health care, this Toolbox was developed by John Hsu, M.D., of Kaiser Permanente and Thomas Rundall, Ph.D., of the University of California, Berkeley. AHRQ funded the research under an IDSRN (Integrated Delivery System Research Network) contract.
Hsu and Rundall found that decisionmakers do not currently use the best available information from management and health services research. The researchers also found that organizational decisionmakers both need and want tools to help:
- Acquire the best available evidence when making management decisions.
- Assess whether evidence is useful.
- Improve the process by which evidence is used in decisionmaking.
The decisionmakers noted the following:
- Little Evidence Use: Decisionmakers rarely use research evidence, in large part because they believe that the available research evidence often is not directly applicable to their decisions.
- Desire for Good Information: Decisionmakers do want the best available information, which they define as information that is accurate, applicable, actionable, and accessible—i.e., the four A's of evidence.
- Diverse Sources of Current Information: Given the perceived dearth of useful research evidence and a broad definition of acceptable information, decisionmakers use many types of available information including more colloquial types of evidence.
- Potential for Tools To Assess Information: Decisionmakers would like tools to help them gather and use evidence, but they also believe that no single tool can address all of their needs. Tools that focus on the common processes of gathering and assessing information for a decision (i.e., tools that help assess the four A's of evidence) could be useful.
Informed Decisions (ID) Toolbox
The Toolbox was created to help bridge the gap between research evidence and organizational decisionmaking. It describes six steps for managers and policymakers to consider when gathering evidence to make a well-informed decision (Table). The ID Toolbox does not attempt to make a decision or determine the right decision for a manager or policymaker. The six steps are:
- Framing the question behind the decision.
- Finding sources of information.
- Assessing the accuracy of information.
- Assessing the applicability of information.
- Assessing the actionability of information.
- Determining if the information is adequate.
These steps are useful whether managers and policymakers are reviewing evidence themselves or are directing consultants and staff to gather relevant information. The Toolbox is available at: http://toolbox.berkeley.edu/.
Six Steps for Managers To Consider for a Well-Informed Decision
Step 1: Framing the Question Behind the Decision
What is the decision and what do I really need to know to make a well-informed decision?
Step 2: Finding Sources of Information
Have I (or my staff/consultant) looked in all the right places for evidence?
Step 3: Assessing the Accuracy of Information—Evaluating Evidence
How much of this information is accurate?
- Is the information valid and reliable?
- Is the information comprehensive?
- Am I missing important perspectives or aspects of my decision?
Step 4: Assessing the Applicability of Information—Evaluating Evidence
Is all of the information applicable for my decision and my organization?
Step 5: Assessing the Actionability of Information—Evaluating Evidence
Is this information adequate for creating an actual plan?
- Which recommendations can I implement?
- Which findings can I convert into concrete steps for implementation?
- Which information provides guidelines about the possible effects of my decision?
- What are the expected effects of my decision?
- What are the possible unintended effects of my decision?
Step 6: Determining if the Information Is Adequate
When do I have enough information to make my decision?
- Is there a single best option?
- Is there more than one reasonable option?
- What should I do if the information I need does not exist yet?
Recommendations for Researchers
Closing the gap between research and practice also requires effort by researchers. In particular, decisionmakers would like evidence that:
- Is more applicable to their actual decisions.
- Includes information on what needs to be done.
- Is more easily accessible.
Moreover, researchers and decisionmakers should consider long-term collaborations to help prospectively identify topics for and parameters of evidence development.
Martelli PF. Evidence-based management [Letter to the Editor]. Harvard Business Review. July/August 2006.
Rundall TG. Evidence-based management: implications for funding health service research. In: Kovner A, D'Aquilla R, Fine D, editors. The Practice of Evidence-Based Management. Chicago: Health Administration Press; forthcoming 2008.
Rundall TG, Martelli PF, Arroyo L, et al. The informed decisions toolbox: tools for knowledge transfer and performance improvement. Journal of Healthcare Management. September/October 2007.
Rundall TG, Martelli PF, McCurdy R, et al. Using evidence when making decisions: views of health services managers and policy makers. In: Kovner A, D'Aquilla R, Fine D, editors. The Practice of Evidence-Based Management. Chicago: Health Administration Press; forthcoming 2008.
Shortell SM, Rundall TG, Hsu J. Improving patient care by linking evidence-based medicine and evidence-based management. JAMA 2007;298:673-6.