Evaluation Utilization – 1.02 Utilization
After all of the careful and difficult thought and work that goes into the planning and implementation of a high-quality evaluation, it is ironically easy and strikingly common for very little thought and effort to go in to the actual use of evaluation findings. This problem is prevalent enough that one of the most often-cited texts in the field of evaluation is Utilization-Focused Evaluation, by Michael Quinn Patton. One might wonder, “What evaluation isn’t focused on use? What is it for if it isn’t going to be used?” Yet many anecdotal reports from program managers and systematic empirical research studies on evaluation use point to the “dusty shelf/drawer problem”—as soon as the evaluation report is completed, it goes directly to a shelf or desk drawer to begin collecting dust. Yet there are many ways that a good evaluation can and should be useful.
The section above on internal and external feedback paves the way for evaluation use; if feedback and reporting are done well, in meaningful and relevant ways, than it is more likely that the evaluation results will effect some change. Visually presenting results in a way that is easily understood by your audience is key. Use graphs, diagrams and other images to depict the story your data is telling. You may have several different audiences for your evaluation results. In each report, use terms that your audience understands. Tailor the level of detail to the audience. Here are some possible ways in which an evaluation might be used:
- Internal reports and discussions about the findings help program administrators and implementers know more about what is and isn’t working in their practice—this can lead to personal, programmatic, and organizational improvements to how the program is implemented
- Unexpected findings can uncover new needs which could be met by creating a new program or by adding new components to the existing program
- Reports to existing funders provide the required accountability that they need to justify present and future funding of the program
- Reports to other stakeholders who occupy prominent positions in the system (such as local legislators) increase community support for the program
- Publicity material (brochures, fact sheets, advertisements) containing evaluation results can foster community support and help reach new participants and forge new partnerships
- Results lead to an evolution of the program and its stage in the evaluation lifecycle—informing the Evaluation Questions that might be addressed and the methodologies used in subsequent evaluation cycles
Note that all of the examples above come about because of evaluation results (whether they were expected or unexpected or about intended or unintended program effects). Yet there are additional useful benefits from evaluation which are related to the process, rather than the results. Such uses are referred to as “process use” and include individual and organizational changes in attitude, thinking, and behavior that result from participating in an evaluation. For example:
- A team of program implementers from a complex program that covers a wide geographical area come together to articulate their program’s theory of change—for the first time—thus arriving at a much better shared understanding of what the program is
- A program implementer identifies stakeholders more intentionally and interacts with them in new ways, gathering helpful information yet also forging new and stronger relationships
- Program staff learn new techniques and gain capacity for gathering and making meaning from pertinent information
- People who once feared and avoided evaluation begin to see it as not only beneficial, but as an integral part of everyday professional practice—a culture of evaluative thinking is fostered