This activity is designed to take place after the final evaluation plan has been completed, but before the evaluation implementation begins. The purpose of this activity to allow participants to experience what it might be like to implement their evaluation plan, what some of the challenges they may face will be, and how those challenges might be addressed, all in a controlled, artificial setting. The idea behind this is that the next best thing to learning by doing is learning by pretending to do.
The activity includes three parts:
- Data collection – the working group will practice collecting data using the measures and methods they have laid out in their evaluation plan and their peers and colleagues can act as stand in participants (or data sources). One of the goals of this part is to bring to light any mistakes or pitfalls in the measures themselves and/or the data collection strategy. Although for newly developed measures you should have already pilot tested them on a sample of participants, this activity allows you one more opportunity to pilot test in a slightly different way and make any final adjustments that seem warranted.
- Data management – the working group will work with the data collected in part 1 and/or fake data generated prior to the activity by the Evaluation Champion to set up a spreadsheet, transcription method, online form or whatever data management strategy is necessary and appropriate given the evaluation plan. The goal of this part is to highlight the need for a detailed data entry and management strategy, and to help bring about a discussion, among the working group members, about how things like missing data will be handled.
- Data analysis – the working group will work with a set of data (either fake or generated in part 1 of this activity) to carry out the analysis strategy laid out in their written evaluation plan. The goal of this part is to demystify the data analysis process and to show that a highly detailed plan for data analysis is only possible to develop once the working group knows what the data will look like.
Once the three main parts of this activity are complete, the working group will take time to discuss and take notes about their experience. These notes and discussion may result in some necessary last minute changes to the evaluation plan, which may end up saving the working group time and effort over the course of the coming evaluation cycle.