Skip Navigation U.S. Department of Health and Human Services www.hhs.gov
Agency for Healthcare Research Quality www.ahrq.gov
Archive print banner

Evaluating Domestic Violence Programs

This information is for reference purposes only. It was current when produced and may now be outdated. Archive material is no longer maintained, and some links may not work. Persons with disabilities having difficulty accessing this information should contact us at: https://info.ahrq.gov. Let us know the nature of the problem, the Web address of what you want, and your contact information.

Please go to www.ahrq.gov for current information.


To improve the health care response to victims of domestic violence, hospitals and health care systems are designing and implementing training, screening, and intervention programs. Formal evaluations of the programs are essential.

The Agency for Healthcare Research and Quality (AHRQ) developed a consensus-driven quality assessment tool for evaluation of hospital-based domestic violence programs. Dr. Jeffery H. Coben, while AHRQ's Domestic Violence Senior Scholar-in-Residence, based the instrument on the views of national experts who took part in an AHRQ-funded Delphi process. The Family Violence Prevention Fund cosponsored Dr. Coben's position.

The experts achieved consensus on 37 performance measures, which have been expanded into a working instrument. The measures are in the form of questions, each with a list of possible responses.

Select to Download Evaluation Tool.


Contents

Background
Tool Development and Testing
Suggested Uses
Guidelines for Completing Instrument
Preparing for Program Assessment
Tool Instructions and Interpretations
Scoring
References
More Information
Download Evaluation Tool

Background

Domestic violence has been recognized as a major public health problem in the United States. Recent studies have demonstrated that 2-4 percent of all women seen in hospital emergency departments have acute trauma associated with domestic violence and another 10-12 percent of women have a recent history of domestic violence.1-3 Similar studies report that 5-15 percent of all women seen in other health care settings have a history of recent domestic violence.4-6 While the majority of injuries sustained by domestic violence victims are classified as superficial contusion, abrasions, and lacerations,3,7 an estimated 73,000 hospitalizations and 1,500 deaths among women are attributed to domestic violence each year.8,9

To address this important problem, health care professionals, victim advocacy groups, and other professional organizations have been working together to improve the health care response to victims of domestic violence. Across the Nation, hospitals and health care systems are designing and implementing domestic violence training, screening, and intervention programs. As these programs develop, formal evaluations of their progress are essential to program planners, participants, policymakers and funders. Unfortunately, instruments to assess these programs have not been available.

Several prospective studies are currently underway to examine the long-term impact of hospital-based domestic violence programs on outcomes of health and safety. While awaiting the results of these studies, it is important that ongoing efforts be directed toward implementing programs of the highest possible quality and establishing measures to evaluate and monitor quality.

The quality of health care and the quality of health-care interventions for domestic violence can be assessed using a health services research paradigm. This approach attempts to measure quality by examining the structure of the health care system, the process of health care delivery, and the outcomes associated with the care provided.10 The paradigm examines the following:

  • Structure—attributes of the settings in which care occurs. This includes the attributes of material resources (facilities, equipment, money), of human resources (number and qualifications of personnel), and of organizational structure.
  • Process—what is actually done in giving and receiving care. It includes the practitioner's activities in making a diagnosis, documenting their findings, and implementing treatment.
  • Outcome—effects of the program on the outcomes of interest, which usually include morbidity, mortality, quality of life, health care utilization, and health care costs.

A fundamental tenet of this approach is the belief that programs with good structures in place will have an increased likelihood of having a good process of care, and good process increases the likelihood of good outcome. While good structures and process of care do not guarantee success, they increase the likelihood of successful outcomes. Similarly, inadequate structures and poor process of care will likely result in poor outcomes.

Most domestic violence initiatives in health care settings have multiple components. Many components are structure- and process-oriented and include:

  • Training health care providers.
  • Establishing a hospital task force or team.
  • Establishing specific policies and procedures.
  • Modifying environments.
  • Screening for victimization.
  • Enhancing intervention services.

Formal measurement of these components will allow administrators, advocates, program managers, and researchers to:

  • Determine how well a program has been implemented at a site.
  • Compare different programs across sites.
  • Determine which program features are most important in producing positive outcomes.

Return to Contents

Tool Development and Testing

The Agency for Healthcare Research and Quality (AHRQ) developed a consensus-driven quality assessment tool for evaluation of hospital-based domestic violence programs. Jeffery H. Coben, M.D., AHRQ Domestic Violence Senior Scholar-in-Residence, based the instrument on the views of national experts who took part in an AHRQ-funded Delphi process. His position was cosponsored by the Family Violence Prevention Fund (FVPF).

Dr. Coben is Associate Professor of Emergency Medicine at Drexel University College of Medicine and Director of the Center for Violence and Injury Control Allegheny-Singer Research Institute, Allegheny General Hospital, Pittsburgh, PA. An authority in prevention of injury and violence, he has published in peer-reviewed literature on identification of domestic violence victims in medical settings.

An earlier version of the tool was first developed and used in a multi-site study evaluating the effectiveness of a training model to improve the emergency department response to battered women.11 The current version was developed with input from a panel of 18 experts, using the Delphi process of consensus development. A separate publication fully describes the details and findings of the consensus development project.12

The instrument is designed to permit a formal assessment of a hospital's performance in implementing a domestic violence program. Panelists were instructed to concentrate on structure and process measures. Performance measures were identified and agreed on by the panel of experts. These measures fell within nine different domains (categories) of domestic violence program activities including:

  1. Hospital Policies and Procedures.
  2. Hospital Physical Environment.
  3. Hospital Cultural Environment.
  4. Training of Providers.
  5. Screening and Safety Assessment.
  6. Documentation.
  7. Intervention Services.
  8. Evaluation Activities.
  9. Collaboration.

The panelists also assigned weights to each of the categories and to each of the individual measures, based on their perceived relative importance. These weights were used to help derive the scoring procedures.

Following completion of the consensus development project, Dr. Coben further operationalized the instrument by putting each performance measure into the form of a question or several questions. In addition, a list of possible responses to each question was developed, categorized, and codified.

Field testing of the instrument was then conducted to determine its feasibility and inter-rater reliability. Reliability testing was conducted at four different hospital sites, using two different pairs of coders. One pair of coders, Dr. Coben and another trained evaluator, were experienced and very familiar with the instrument, while the other pair of coders were inexperienced and received minimal training on administering the instrument. In all cases, inter-rater reliability was very high (Kronbach's alpha ranging from 0.97 to 0.99 in the experienced coders and 0.96 to 0.99 in the inexperienced coders).

Return to Contents

Suggested Uses

The instrument can be used for a variety of purposes:

  • The measures can serve as useful benchmarks or objectives for program achievement.
  • An individual site's performance can be assessed repeatedly over time to determine progress in program implementation. Ideally, sites should perform a baseline assessment prior to implementing a new program and re-assess their status annually.
  • The instrument can be used by researchers and program administrators to compare and contrast different programs across different sites.
  • If linked to appropriate outcome measures, the instrument could be used to help determine which program features are most important in creating positive long-term outcomes for domestic violence victims.

The focus of this instrument is on hospital-based domestic violence programs. While domestic violence programs have been implemented in many other settings, the generalizability of the instrument to other settings, including other health care settings (such as private physician offices or outpatient clinics), has not been tested.

Return to Contents

Guidelines for Completing Instrument

The evaluation instrument can be self-administered or administered by an independent evaluator in conjunction with a representative from the hospital domestic violence program. In either situation, the individual who is most familiar with the domestic violence program should participate in the process. The required information can be best obtained via a "site visit" in the hospital. The hospital should have sufficient time to assemble the documents to be reviewed. Three weeks advance notice is suggested. The assessment procedures should include a review of documents as well as a physical tour of the facility to examine posters, brochures, documentation procedures, equipment (i.e., cameras), and other supplies. Approximately 4 hours should be allocated for completion of the instrument.

Return to Contents

Preparing for Program Assessment

To assist in assessment of the hospital's domestic violence program, the materials below should be assembled and reviewed. The hospital site should be given sufficient time to gather the information (usually 3 weeks). The site should be asked to flag or mark the information if it is part of a larger document, such as a hospital policy manual.

Suggested documentation:

  • All written hospital policies, protocols and procedures regarding domestic violence.
  • Relevant department-specific policies and procedures regarding domestic violence (such as Emergency Department, Security Department, obstetric/gynecologic).
  • Information on domestic violence in the hospital's Employee Assistance Program.
  • Documentation of the activities of the hospital's domestic violence task force or committee including:
    • Roster of participating individuals, departments, and agencies.
    • Schedule of meeting dates.
    • Prior meeting minutes or notes.
  • Materials used and/or distributed in any domestic violence training for hospital staff.
  • Schedules of planned trainings or strategic plans for training employees.
  • Forms or checklists used for domestic violence programs including:
    • Domestic violence screening forms.
    • Standardized documentation forms.
    • Consent-to-photograph forms for domestic violence cases.
    • Intervention checklists for staff to use when victims are identified.
    • Standardized safety assessment forms.
    • Referral forms.
  • Information on prior evaluations used as part of the program including:
    • Assessments of staff attitude and knowledge of domestic violence.
    • Prior chart audits to assess for domestic violence screening.
    • Other documented evaluation procedures.
  • Documentation of hospital preventive outreach and public education on the topic of domestic violence.
  • Information on financial resources that the hospital provides for the domestic violence program.
  • Copies of brochures, pamphlets, or referral cards for victims of domestic violence or available to the public in the hospital.

In addition, the site should conduct a chart review to determine compliance with domestic violence screening. The site should review a random sample of charts from an area of the hospital where domestic violence screening is supposed to be conducted. The percentage of female patients who were actually screened for domestic violence, based on chart documentation, should be recorded.

Return to Contents

Tool Instructions and Interpretations

Category 1. Policies and Procedures

1.1. This information should be obtained from a direct review of the hospital-wide policies and procedures manual.

1.1(a). Simply requires a definition. Does not require that the definition meet an external standard or commonly used definition.

1.1(b). If any new and/or existing employees are required to obtain training, this will result in a "yes" response. If staff is encouraged, but not required to obtain training, this will result in a "no" response.

1.1(c). Universal screening refers to the screening of all women seen in a clinical setting. If the screening policy includes both men and women, this will result in a "yes" response. If the screening policy includes one clinical setting (e.g. Emergency Department) or more than one clinical setting, this will result in a "yes" response. Note, this question does not require a "mandate," but does require that the hospital encourage, support, and/or advocate universal screening.

1.1(d). Self-explanatory.

1.1(e). Assess if the hospital policy provides information and/or instructions on the documentation required for victims of domestic violence.

1.1(f). Assess if the hospital policy includes information and/or instructions dealing with the referral of victims of domestic violence. Note, a specific domestic violence agency or hotline number does not need to be included in the hospital policy.

1.1(g). Assess if the hospital policy provides information dealing with the legal obligation of health care providers to report domestic violence to other authorities.

1.2. A "domestic violence task force" could be called other things, such as a "Working Group," "Abuse Committee," or "Domestic Violence Coordinating Committee." To qualify as a "yes" response, there needs to be evidence that the group focuses on the topic of domestic violence, either exclusively or in conjunction with other closely related topics (e.g., sexual assault, child abuse). This information should be obtained through discussions with the site representative and by reviewing the following types of material:

  • Roster or mailing list of representatives on the task force.
  • Schedule or announcement of meeting dates.
  • Prior meeting notes or minutes.

1.3. "Direct financial support" refers to internal funding, provided by the hospital, for materials, supplies, and personnel. This does not include extramural grants or contracts for the program or "in-kind" contributions such as space, electricity, and heat. This does include salary and fringe benefits, if the hospital covers some or all of these costs for the domestic violence program coordinator or other personnel directly involved in the program.

1.4. Self-explanatory. Compare with 1.1(c). Requires a written mandate for screening.

1.5(a). "Regular chart audits" means that at least a sample of charts are reviewed on a regular basis (at least quarterly) to determine if screening is being performed or documented.

1.5(b). "Positive reinforcers" would include awards or rewards for screening, contests identifying the individuals or departments with the highest screening rates, direct feedback to clinicians who are screening, and prompts or signs to encourage screening.

1.5(c). "Punitive measures" would include the enforcement of corrective and/or disciplinary actions to individuals or departments if screening does not occur.

1.6. A "yes" response requires that specific written procedures be identified. These procedures may be found within the security department, the hospital's policies and procedures manual, the hospital's domestic violence policies, or other similar materials.

1.7. The domestic violence coordinator could be a hospital employee or employed by an outside agency. Typically, the domestic violence coordinator would be responsible for managing the hospital's domestic violence task force, coordinating training activities, scheduling meetings and providing leadership for the programmatic activities.

Category 2. Physical Environment

2.1. This information is obtained from direct observations throughout the hospital facility. Each separate location housing the public display of domestic violence-related posters or brochures counts as one location. List the total number of locations, up to 35. Separate rooms within the same location (e.g., waiting room, triage area, and patient treatment room in the emergency department) would each count as a separate location. Multiple posters or posters and brochures in one room (e.g., 3 posters plus brochures all located in the emergency department waiting room) would count as only one location.

2.2. Same as 2.1, except material must include referral information (hotline numbers).

2.3. This information relates to how and where a victim is cared for in the hospital while awaiting placement in a community-based shelter, and it assumes that the victim does not require admission to the hospital for medical reasons. The information may be obtained from a review of hospital domestic violence-related policies and procedures or from direct conversations with the domestic violence coordinator.

Category 3. Cultural Environment

3.1. Requires documentation of a previous written assessment (i.e., survey or questionnaire) of staff knowledge and attitude about domestic violence.

3.2. A good indicator of program start-up would be the date of the first task force meeting.

3.3. Information should be obtained from a review of hospital policies and procedures, written information from the Employee Assistance Program, and/or discussions with the domestic violence coordinator.

3.4. Examine policies, training materials, and referral information for evidence of cultural competency as described.

3.5. Self-explanatory. Documented participation should include more than sponsorship.

Category 4. Training of Providers

The information for this section should be obtained from review of available training materials and discussion with the domestic violence coordinator. If the domestic violence coordinator is not directly involved with the training program, he or she should provide this information in consultation with the training staff.

4.1. A "formal training plan" indicates that training is systematic and ongoing, rather than haphazard and irregular. Evidence of strategic planning for systematic training should be provided. "Clinical staff" refers to staff that have direct patient care responsibilities. "Non-clinical staff" would include administrators, clerical staff, security staff, and others without direct patient care responsibilities.

Category 5. Screening and Safety Assessment

5.1. Use of a "standardized instrument" should be assessed by actually examining the instrument itself. The screening should include at least three questions to qualify as a "yes" response, but these questions can include "framing questions" about domestic violence as well as specific questions on abuse. Determine if the instrument is a separate "stand alone" form or if the questions are routinely incorporated/embedded in the clinical record, for example, as part of the nursing assessment form or part of the physician's review of systems history form.

5.2. This assessment requires that a sample of charts be retrieved and reviewed for screening documentation. Charts reviewed should be of female patients who have visited a clinical setting where screening is supposed to be conducted. A random sample of a minimum of 25 charts is suggested.

5.3. A "standardized safety assessment" means that all patients identified as in an active abusive relationship are asked a series of routine questions regarding their personal safety. These questions may be in the form of an assessment instrument (e.g., Danger Assessment) or may be verbally administered by the individual(s) responsible for assisting victims. The safety assessment should be standardized, meaning all victims are asked the same series of questions.

Category 6. Documentation

6.1. This information should be obtained by reviewing the actual documentation form or instrument, if one exists.

6.2. Information on procedures for photography should be obtained from the domestic violence coordinator and from clinical staff responsible for photographic documentation. The camera should be readily available.

6.2(d). The consent-to-photograph form should be examined. To qualify as a "yes" response, this form should specify that it provides consent to photograph injuries from abuse.

Category 7. Intervention Services

7.1. A "standard intervention checklist" refers to a readily available reference for clinical staff to use that provides information on the required/suggested steps to be taken if a victim is identified. This checklist may be in the form of a clinical algorithm, a "pull-packet," a computerized reminder system, or other similar format. If the checklist is embedded in the hospital policies and procedures manual, it should be readily accessible for immediate staff use to qualify as a "yes" answer.

7.2. "Victim advocacy services" refers to service provision above and beyond immediate safety assessment and referral. Advocacy services usually include the provision of empathic support, empowerment and options counseling, education regarding the cycle of violence, promoting self-efficacy, establishing linkages with community resources, and maintaining followup if desired by the victim. A "trained victim advocate" is someone who has received specific training in providing the above services to victims of domestic violence.

7.3. Examples might include assessments for depression, post-traumatic stress disorder (PTSD), and other mental health problems. Are these types of assessments performed as a part of the evaluation/intervention for victims of domestic violence?

7.4. Does the hospital provide transportation for victims if needed, or vouchers for transportation (e.g., cab slips)?

7.5. A "yes" response would indicate that victims are offered followup domestic violence support/counseling with personnel from the hospital domestic violence program. Referral to another site (i.e., shelter program or another support group) would not qualify as a "yes."

7.6. On-site legal counseling would include activities such as:

  • Explaining victim's legal rights and options.
  • Providing information on how to obtain a restraining order or protection from abuse order (PFA).
  • Providing assistance with obtaining a restraining order or PFA.

7.7. To qualify for a "yes" response, the program would need to offer some service above-and-beyond routine child/adolescent health services. Examples might include specific assessments for children who witness domestic violence, assessments for co-occurring child abuse and domestic violence, or providing child care during maternal followup domestic violence counseling visits.

7.8. "Evidence for coordination" could include:

  • A hospital domestic violence task force that includes mental health and/or substance abuse professionals.
  • Policies or procedures that include screening domestic violence victims for mental health (depression, PTSD, etc.) concerns and/or substance abuse.
  • Routinely assessing sexual assault victims for domestic violence and routinely assessing domestic violence victims for sexual assault.
  • Other similar activities.

Category 8. Evaluation Activities

8.1. To qualify for a "yes" response, there should be some evidence that the program collects and analyzes data on a regular basis. The data may be qualitative or quantitative and it may include:

  • Evaluations of the domestic violence training activities.
  • Assessment of staff knowledge and attitudes about domestic violence.
  • Interviews or focus groups with providers on the topic of domestic violence services.
  • Assessments of program implementation.
  • Chart audits to assess for domestic violence screening and documentation.

8.1(a). A program demonstrating evidence that charts are audited to assess for domestic violence screening would receive a "yes" on this item, in addition to a "yes" on item 8.1.

8.1(b). Peer-to-peer reviews involve professionals discussing specific cases of domestic violence in an effort to improve the quality of the program. A program demonstrating evidence of peer-to-peer reviews would receive a "yes" on this item, as well as a "yes" on item 8.1.

8.2. Standardized feedback differs from peer-to-peer reviews, in that standardized feedback can be provided by the domestic violence coordinator, advocate, or other individuals involved in the program. Feedback on domestic violence screening performance, the provider's chart documentation, or the outcome associated with newly discovered cases of domestic violence are examples of activities that would qualify for a "yes" response.

8.3. Self-explanatory.

Category 9. Collaboration

9.1-9.2. For these items, the hospital should provide evidence of collaboration with the agencies indicated. Local domestic violence programs generally refer to shelter programs and/or advocacy agencies. Local law enforcement could include police, district attorney's office, sheriff, or other law enforcement agencies. Participation on the domestic violence task force can be documented via a roster or mailing list of task force participants. Other specific examples of collaboration in the areas mentioned should be sought.

9.3(a). Examples of collaboration within the same health care system might include:

  • A hospital domestic violence program that also involves and collaborates with an affiliated, off-site, out-patient clinic.
  • Coordination between a hospital-based domestic violence program and affiliated off-site private physician offices.
  • Collaboration between two or more hospitals that are part of the same health care system.

9.3(b). Examples would include collaboration between hospital domestic violence programs that are in the same service region but not part of the same parent health care system. If a hospital collaborated with multiple hospitals in their region, including some within the same health care system and some in different health care systems, they would receive a "yes" for both 9.3(a) and 9.3(b).

Return to Contents

Scoring

Scoring procedures are based upon a weighting scheme developed with input from the Delphi panelists. For each of the nine categories, the site can receive a score ranging from zero to 100 points, with 100 being the best possible score. The total (raw) score for each category is simply a sum of the points received for each question within that category. The point values assigned to each possible response are noted in parentheses next to the response. It should be noted that in some cases only a single (forced choice) response is permitted (e.g., question 1.3), while other questions allow for multiple positive responses (e.g., question 1.1).

To calculate a single total score for the entire instrument, the following procedure should be followed:

Raw Score for Category 1 = ___ multiplied by 1.16 = A
Raw Score for Category 2 = ___ multiplied by 0.86 = B
Raw Score for Category 3 = ___ multiplied by 1.19 = C
Raw Score for Category 4 = ___ multiplied by 1.15 = D
Raw Score for Category 5 = ___ multiplied by 1.22 = E
Raw Score for Category 6 = ___ multiplied by 0.95 = F
Raw Score for Category 7 = ___ multiplied by 1.29 = G
Raw Score for Category 8 = ___ multiplied by 1.14 = H
Raw Score for Category 9 = ___ multiplied by 1.04 = I

Total Score = A+B+C+D+E+F+G+H+I divided by 10

The total score is also based on a scale of 0-100, with 100 being the best possible score. Higher scores would indicate a more fully implemented program. At the current time there are insufficient data to provide cut-off scores for what may constitute an acceptable, exemplary, or ideal program. Rather, it is suggested that sites use these scores to establish their own baseline and monitor their progress over time. As more data becomes available, these data will be posted so that individual sites can compare their results with the distribution of scores across multiple sites.

Return to Contents

References

  1. Dearwater SR, Coben JH, Campbell JC, et al. Prevalence of intimate partner abuse in community hospitals. JAMA 1998;280:433-8.
  2. Abbott J, Johnson R, Koziol-McLain J, Lowenstein SR. Domestic violence against women: incidence and prevalence in an emergency department. JAMA 1995;273:1763-7.
  3. Muelleman RA, Lenaghan PA, Pakieser RA. Battered women: injury locations and types. Ann Emerg Med 1993;28:486-92.
  4. Jones AS, Campbell J, Gielen A, et al. Annual and lifetime prevalence of partner abuse in a sample of female HMO enrollees. Women's Health Issues 1999;9:295-305.
  5. McCauley J, Kern D, Kolodner K, et al. The "battering syndrome": prevalence and clinical characteristics of domestic violence in primary care internal medicine practices. Ann Intern Med 1995;123:737-46.
  6. Gazmararian JA, Lazorick S, Spitz AM, et al. Prevalence of violence against pregnant women. JAMA 1996;275:1915-20.
  7. Coben JH, Forjuoh SN, Goldolf EW. Injuries and health care use in women with partners in batterer intervention programs. J Fam Violence 1999;14:83-94.
  8. Crowell NA, Burgess AW, editors. Understanding violence against women: panel on research on violence against women, National Research Council. Washington, DC: National Academy Press; 1996.
  9. Tjaden P and Thoennes N. Full report of the prevalence, incidence, and consequences of violence against women. Washington DC: National Institute of Justice and the Centers for Disease Control and Prevention; November 2000. Also available at: http://www.ncjrs.org/txtfiles1/nij/183781.txt. Exit Disclaimer
  10. Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q 1966;44(3 suppl):166-206.
  11. Campbell JC, Coben JH, McLoughlin E, et al. An evaluation of a system-change training model to improve emergency department response to battered women. Acad Emerg Med 2001;8:131-8.
  12. Coben, JH. Measuring the quality of hospital-based domestic violence programs. Acad Emerg Med. In press, November 2002.

Return to Contents

More Information

For more information on the Delphi Instrument for Hospital-based Domestic Violence Programs, contact:

Jeffrey H. Coben, M.D., Director
Center for Violence and Injury Control
Allegheny General Hospital
320 E. North Ave.
Snyder Pavilion, Suite 214
Pittsburgh, PA 15212
Phone: (412) 359-6260
Fax: (412) 359-6261
E-mail: jcoben@wpahs.org

Return to Contents


Download Evaluation Tool

Your browser may support downloading the evaluation tool from this Web page. To download it, right click on the links and then select "Save Target As" (Internet Explorer) or "Save Link As" (Netscape):

Return to Contents

Current as of September 2002

 

The information on this page is archived and provided for reference purposes only.

 

AHRQ Advancing Excellence in Health Care