In many cases, the individual(s) who collect the data are not members of the working group. Sometimes data collectors are not even staff members of the organization conducting the evaluation. One example of this is an evaluation based on observations made by classroom teachers where the classroom teachers are not actually staff of the organization sponsoring the program/evaluation. Regardless of whether the data collectors are part of the evaluation team or are external, it is essential to provide clear instructions and training on how to collect the data accurately and consistently.
When planning for this type of training and/or training document, it is best to err on the side of comprehensive instruction rather than assuming that various individuals will collect data in the same way. Even data collectors who are simply handing out a “self-administered” survey should follow the same guidelines or script for how to introduce the survey to the respondents.
General areas of consideration for data collection training include:
- How and when instructions are given to respondents
- How and when informed consent is obtained from respondents
- What guidelines are needed for subjective assessments (observation, rubrics, etc.)
- What definitions of terminology are needed
- Do related tasks need to be completed in a certain order
- What should be done with data once it is collected
- What time limits are there for respondents to complete the instrument?
If possible, data collectors should participate in the piloting process so that they have practice implementing the measures. Often pilot testing contributes to the development of training materials because it can be difficult to predict the nuances of implementing evaluation measures. If feasible, the working group should meet with data collectors after pilot testing, before any training materials are created; and then again after so that data collectors can provide feedback on the materials. It is also advisable to meet with data collectors regularly throughout the data collection process so that any questions and/or inconsistencies can be addressed immediately.