Evaluation Planning – 2.02 Program Review
The goal of this step is to gain a firmer understanding of the components and characteristics of the program and its parent organization, including how the program operates and whom it serves. The output will be a fairly comprehensive program description, which will probably be more helpful than the benchmark initial description given in the earlier Evaluation Capacity Surveys. This is an iterative step – any tools that have been used or developed in previous steps in order to describe the program should be discussed and reviewed – such as the surveys and stakeholder map. It is also appropriate to examine any curricula, guidelines, manuals or prior evaluations that might exist in relation to the program.
The following questions may help guide the discussions:
- Why does the program exist?
- What is the program’s mission or vision?
- What are the program’s activities?
- Who participates in these activities?
- When does the program take place?
- Where does the program take place?
Even though these questions are listed from the broadest to the more specific, that is not necessarily the best order to follow when working with a group. Sometimes the best place to start is with what program staff actually do – brainstorm the activities of the program – and then build from there to the more general issues of program mission/vision. This may be best if the program is underway or well-established and the working group includes active program managers. For programs that are being developed, or that are especially mission-driven, it may work best to start with broad themes and work toward discussing specifics. Regardless of the sequence, the discussion should be inclusive and ideally should help build relationships within the working group and with the Evaluation Champion.
This step should be approached knowing that some of these questions can probably never be completely answered to everyone’s satisfaction. There may be some areas of disagreement, or responses for which answers are uncertain, but it is not necessary to have complete agreement at this point. The descriptions outlined should be fairly comprehensive so as to “fill out” a relatively complete picture of the elements of the program. This effort to define the program can be surprisingly difficult. This is an important discussion to have before moving on to the next step, which will be to set some boundaries for the program.
In our experience this step has been an exciting one for many programs. It offers staff the opportunity to step back and reflect on ALL that goes into their program. Sometimes some unexpected connections arise as they recognize the contributions of program components they may previously have undervalued. When you’re doing this step it is probably a good idea to use a white board or black board and draw things out as the group is saying them.
The working group shouldn’t get ahead of themselves —don’t start drawing the logic or pathway models yet. Stay focused on encouraging brainstorming. The next few steps will give shape to what is generated in this step. The result of this step will typically be a tangled mess that shows the level of detail in the program. The complexity of the program may even surprise many in the working group, who previously may have had a rather narrow, organized view of the program.