2.09 Reflection and Synthesis
The purpose of this step is to take stock of where the working group is in their thinking about the program and its evaluation, review and revise the work done so far and, if possible, to interview stakeholders so that their opinions can also be integrated.
It would be a rare program indeed that goes through the steps in the Protocol in sequential order one time and gets it “right” or to a level with which they are satisfied. If this process is working correctly, when they get to this synthesis step the working group should immediately see things from earlier steps that they want to go back and revise or rework. In addition, when possible, interviewing stakeholders can shed new light on the program and can inform revisions to the established program boundaries and model (see Stakeholder Interview Activity). However, this process is not always feasible. Therefore, it is important that the working group continue to keep stakeholder opinions in mind throughout the revision process.
The synthesis step actually occurs throughout the entire process, but we place the step here so that you deliberately stop at this point to assess the entire program modeling picture. Another way to think of this synthesis step is to contrast a step-by-step process with a more dynamic and adaptive one. If you look at the steps in the planning stage of the Protocol you might get the impression that the way to do evaluation planning is to take each step in order and then be done. In fact, what we want to encourage is a more dynamic approach to planning that cycles through these steps several times – and not even necessarily in the same order – until the working group is satisfied that they have developed a high-quality model and evaluation scope appropriate for their purposes.
When does this process end? In one sense it never does. In another sense, we have to be practical and recognize that the goal is to get the best model for our purposes within the time that we have available. So, ideally the working group will decide to end the process at some point that makes sense for their work – and recognize that this is a process they should revisit from time to time.
When the working group has settled on a program model that they will use going forward in this process, they may choose to do a more formal review using the structured feedback document (Logic Model and Pathway Model Feedback Form) provided in Appendix XV of this Guide. A Structured feedback can help to provide an “outside” perspective on the program logic and pathway models.
One of the key products of this synthesis step would be a revised program description that reflects new thinking about program boundaries, stakeholders and outcomes. The thinking done at this step in the Protocol will also contribute to a concise evaluation purpose statement, which is the second step in the evaluation planning phase.
At this point the working group will have the compiled set of materials that reflect the work done in this stage of the protocol, including the stakeholder map, program logic model, pathway model, boundary analysis, lifecycle analysis from the perspectives of key stakeholders, evaluation scope and system links.