Activity: Question-Claim Match-Up, Stakeholder Review
Once the working group has a set of draft questions developed, there are some quick activities that can help narrow the list down, and ensure that the questions are appropriate and of high quality.
The first activity is a “question-claim match-up.” This activity simply consists of brainstorming the possible claims one could make after collecting data to address the question at hand. For example, if the question were “Is the Earth round?” the claim you would hope to make is that either A) the Earth is round or B) the Earth is not round. This exercise may help working group members identify how the language and phrasing of their question may affect their methodological choices and the overall usefulness and appropriateness of their evaluation effort. This activity should be done as a group, as brainstorming about claims may bring up differences in perspective and opinions that should be settled before moving forward with the evaluation plan.
The second activity is a peer or stakeholder review. If possible, invite stakeholders and/or peer program implementers to take a look at the developed evaluation questions. The following prompts may help guide their feedback.
- What sort of claims would be possible, if the evaluation yielded favorable evidence for the evaluation question?
- Does the evaluation question fit the Evaluation Purpose Statement? How/how not? Any suggestions?
- Comment on the alignment between this evaluation question and the program’s lifecycle stage; does the evaluation question make sense for “where” this program “is”?
- Does the evaluation question clearly relate to the Program Model? If so, how? If not, what’s missing?
- Does the evaluation question make clear:
– exactly what is being measured (the key constructs)?
– the group(s) to whom the question applies?
– (if appropriate) what the basis for comparison will be?