Skip Navigation Archive: U.S. Department of Health and Human Services www.hhs.gov
Archive: Agency for Healthcare Research Quality www.ahrq.gov
Archive print banner
Assessment of Self-Evaluation Training for the Medical Reserve

This information is for reference purposes only. It was current when produced and may now be outdated. Archive material is no longer maintained, and some links may not work. Persons with disabilities having difficulty accessing this information should contact us at: https://info.ahrq.gov. Let us know the nature of the problem, the Web address of what you want, and your contact information.

Please go to www.ahrq.gov for current information.

Stages of Program Development and Evaluation

Given the limited resources and time available to most MRC unit coordinators, program evaluation activities should be as practical and focused as possible. This will help assure that the evaluation is conducted efficiently and addresses the issues of greatest concern to stakeholders.

Too often, evaluation is perceived as an activity reserved for mature programs that want to examine their overall effectiveness. Although this form of evaluation, known as summative evaluation, can be very valuable, entities like the MRC can also benefit from evaluation activities that support program improvement (i.e., formative evaluation) while the program is still growing. Thus, one approach unit coordinators can use to promote a realistic and practical evaluation strategy is to link evaluation activities to various stages of program development.

The published literature characterizes the development of non-profit or volunteer-based organizations in various ways. This development is often referred to as the "life cycle" of an organization, since it tends to progress through a series of defined stages that are similar to the maturation process a person goes through from infancy, to childhood, adolescence, and adulthood. The defining activities and structures of one stage are not necessarily going to be the same as those in another stage.4

One of the simplest and most effective ways to view organizational development is through the following four stages:5

  • Stage #1: Formation—the organization or coalition is young and the basic strategy is one of survival. The structure is highly individualized.
  • Stage #2: Implementation—the organization or coalition is growing and the strategy is to establish systems to manage the growth and maintain control. The organization has become functionally structured at this point, which may be characterized by the establishment of a governing body or steering committee, development of an action plan, and proactive recruitment.
  • Stage #3: Maintenance—the organization or coalition is fully established and the strategy is geared towards expansion and diversification. The structure is fully developed to enable the organization to exist as a full partner with other organizations.
  • Stage #4: Outcome—the organization or coalition has achieved its primary goals and the strategy is to seek additional diversification of services as some initial services reach the end of their life cycle.

Alternative models of program development may specify slightly different stages; however, there are generally common elements between them. For instance, most models characterize the initial stages of development as highly dependent on innovative or entrepreneurial activities, which seek to develop a market niche or build on a particular ideology. A primary driver of initial development is the acquisition of resources and building of strategic relationships. Similarly, mature organizations typically have achieved some level of institutionalization, which occurs when policies and rules become more firmly established and formalized.6

These stages provide a conceptual framework for program development and can be useful to the MRC coordinator in planning to assess their unit's performance. Just as the activities and structures in one stage will not necessarily be the same as in another stage, so too may the evaluation criteria differ. MRC coordinators conducting a self-evaluation must ensure that the evaluation goals are defined to properly match the focus of the organization at that point in time.7 It is also important to specify at the outset, both to internal and external stakeholders, why the evaluation is being done and how the information will be used.8

Using the four stages of development previously described, we examine some of the evaluation goals that might be most applicable. During the formation and implementation stages, program activities are untested and the goal of evaluation is to refine plans. Evaluation activities may focus primarily on making sure the right stakeholders have been engaged and that appropriate and realistic goals and objectives have been established. In addition, such issues as resource acquisition and the development of external partnerships may be examined. The governing body or steering committee for the MRC unit should review any goal(s) and objectives outlined in a strategic plan to ensure they are aligned with the needs of the community and the expectations of key partners and stakeholders.

During the implementation stage, activities will be field-tested and modified. The goal of evaluation is to characterize real, as opposed to ideal, program activities and improve operations, perhaps by revising plans and refining internal process issues.9 As the MRC unit progresses into the maintenance stage, it may seek to expand its activities and service offerings. Performance measurement will focus primarily on process measures to determine if the unit is operating as intended. It may also be appropriate at this stage to examine the potential impact of activities on short-term outcomes (e.g., changes in knowledge or skills). In this way, measures of effectiveness may shift towards goal attainment and productivity.

As the MRC unit enters the outcome stage, it will have accumulated significant experience and data to determine whether it has been successful in meeting (or making progress towards meeting) its goals and objectives. During this stage, enough time has passed for the program's effects to emerge and the goal of evaluation should be to identify and account for both intended and unintended effects. A results-oriented approach will shift the focus primarily on outcomes measures, with the understanding that this information should feed back into strategic planning to reexamine and modify the unit's goals and objectives, as necessary. This will help MRC coordinators ensure that as their unit grows, it remains responsive to the most pressing needs of the community.


4. Robert E. Quinn and Kim Cameron. "Organizational Life Cycles and Shifting Criteria of Effectiveness: Some Preliminary Evidence." Management Science, Vol. 29, No. 1 (January 1983), pp. 33-51.
5. The Centers for Disease Control and Prevention, Building Partnerships. http://www.cdc.gov/drspsurveillancetoolkit/docs/BUILDING_PARTNERSHIPS_TO_MARKET_THE_MESSAGE.pdf
6. Robert E. Quinn and Kim Cameron. "Organizational Life Cycles and Shifting Criteria of Effectiveness: Some Preliminary Evidence." Management Science, Vol. 29, No. 1 (January 1983), pp. 33-51.
7. Ibid.
8. Marty Campbell and Charles McClintock. "Shall We Dance? Program Evaluation Meets OD in the Nonprofit Sector." OD Practitioner, Vol. 34, No. 4 (2002). Available at: http://www.irvine.org/assets/pdf/pubs/evaluation/Shall_We_Dance.pdf.
9. The Centers for Disease Control and Prevention. Framework for Program Evaluation in Public Health. MMWR 1999;48(No. RR-11). http://www.cdc.gov/mmwr/PDF/rr/rr4811.pdf


Return to Contents
Proceed to Next Section

 

The information on this page is archived and provided for reference purposes only.

 

AHRQ Advancing Excellence in Health Care