3.01 Evaluation Plan Introduction
The goal of this step is to present and discuss the purpose and components of an evaluation plan. The working group should understand the work that is entailed, and the importance both internally and externally of having a fully articulated and written plan.
A written evaluation plan captures important program and evaluation information and guides the implementation of the evaluation, which is especially important if there are new staff who will be doing some of the work. In addition, having a written evaluation plan can help to establish credibility with stakeholders – especially funders – because it conveys information to them and is a demonstration of the commitment to quality evaluation.
The written evaluation plan developed by users of this guide will likely include section headings that map onto many of the steps in the Protocol. The written plan should be formatted so that sections describing the program and its boundaries appear early in the document. Sections titled “Measures” “Sampling” and “Analysis” (for example) should appear later in the document. In addition to the written sections of the plan, a complete document will include several appendices, including a copy of the logic and pathway models, a stakeholder diagram, and copies of any intended measures.
Several of the outputs from Stage 2 will be used to develop the written evaluation plan. Typically we include the program mission statement and program description, the program logic model and pathway model (perhaps as appendices or attachments), and stakeholder map. Decisions about evaluation scope from the previous stage will inform decisions made within the evaluation plan. The plan will also include new products– the evaluation purpose statement, the evaluation questions, a description of the proposed sample, the proposed measures, the proposed design, a data collection and management plan, the plans for data analysis, plans for evaluation reporting and utilization, and an evaluation timeline. These new components are the classic elements of any good research plan, covering the objectives and methodology to be used.
A key distinction in evaluation planning is differentiating between an evaluation cycle and the evaluation of the program over its entire life. An evaluation “cycle” is the evaluation that you do over a specific period of time. Many organizations use an annual evaluation cycle. The endpoint of an evaluation cycle is often determined by when the organization and program are doing major reports. Again, many organizations find it convenient to do annual reports, which is a tip-off to the notion that they are on an annual evaluation cycle. When the working group developed the evaluation scope in Step 2.08, they were essentially determining the extent of the program evaluation they were going to take on in the next evaluation cycle (e.g., over the next year). An evaluation plan is typically written in terms of the next evaluation cycle, describing, in detail, how the staff will evaluate their program over that time period.
We have already seen that programs evolve through phases over time. Evaluation needs to change and adapt so that it is always appropriate (that is, symbiotically linked) with where the program is in its development. This means that over the life of a program there are likely to be multiple evaluation cycles and the evaluation plan for each cycle will change. We will keep coming back to this idea in the sections below.