Evaluation Planning – 1.02 Memorandum of Understanding/Agreement
The Evaluation Champion and relevant decision maker(s) will need to negotiate, identify and outline the responsibilities and expectations of the Evaluation Partnership members and put the finalized commitments into a written Memorandum of Understanding (MOU). Why is a written MOU important? First, in order to develop such a product, the group needs to reach consensus about the roles of the key participants and the expectations of each. While this could be done implicitly without setting it out in written form, we assume that an evaluation is a dynamic and evolving entity and that such a document can help the participants to remain focused on the nature of what they have undertaken. While the understanding of roles and expectations may evolve over time – and the MOU can be revised accordingly – without such a document it is likely that there will be greater misunderstanding and confusion over time. Second, a written MOU is itself a form of feedback. It signals to the entire organization what is happening. It encourages various stakeholders to react to it, to state any interests or concerns they might have up front. And, it helps the stakeholders to adapt to the changing circumstances that any evaluation necessarily introduces. Finally, an MOU is a way to be transparent. Without it, different organizational stakeholders might tend to misinterpret or misunderstand the nature of the work. With it, the organization has an objective written statement that describes who will be responsible for what in the evaluation.
The development of the MOU may take as long as a couple of weeks, depending upon the number of people involved and the scope of the work envisioned. The discussion (and eventual document) should identify the working group members by name and specific role. In addition, the MOU should detail the goals for the project, expectations for the Evaluation Champion, working group members, organization administrators, and organization staff.
There are several key points to negotiate at this time:
Project Goals. The goals of the project should be agreed upon as part of the MOU. If the primary focus is to plan and implement an evaluation for a single program, then the scope of work is smaller. If this is viewed as part of a larger process of building evaluation capacity and evaluation culture within the organization, then more time would be required for trainings, communication, broader staff meetings, and so on.
Who will be in the evaluation working group? Deciding who will be part of the working group that works directly with the Evaluation Champion is an important consideration. The Evaluation Champion can work with one or more working group members to train them on aspects of evaluation so that they become an evaluation resource to the organization.
Responsibilities and roles of the EP members. How will the key decision maker(s) support the evaluation working group and Evaluation Champion? What expectations do the key decision makers have of the evaluation working group and Evaluation Champion reporting back to them?
Time commitments.
General timeline of the expected process for a quality evaluation. Depending on priorities and time availability, it may take several months — even up to 6 months or more — just to create the evaluation plan, then more time to implement the plan and analyze the results. The time put into planning will pay off in the long run.
Time commitments of staff and working group members. The members of the working group are typically comprised of staff members who are directly engaged with the program and are committed to evaluation. The working group members are the core team that works with the Evaluation Champion. Additional staff members may also assist at various points in the process. There will be at least one meeting with stakeholders, and several meetings with the working group members. There will be a need to educate the working group and other staff about evaluation, and eventually about the specifics of program modeling, measure development, and the like. These meetings will have to occur within the busy and demanding schedules of program staff. The time commitment of working group members will vary depending on how much support and collaboration the Evaluation Champion desires.
Time commitments of the Evaluation Champion. The Evaluation Champion’s role and responsibilities will be shaped by the project’s goals and scope of work, and by the availability of and roles assigned to members of the working group. In cases where the Evaluation Champion is a staff member internal to the participating organization, this individual will be spending additional hours planning meetings and addressing questions. The additional responsibilities of the Evaluation Champion should be considered and planned for in light of their existing duties and this should be documented in the MOU.
Policy regarding evaluation. Although many organizations have some sort of policy stating that evaluation is a requirement, most organizations don’t think about the amount of time that this will take. Ten to fifteen percent of an employee’s working time is not unreasonable. We encourage organizations to make their commitment more explicit by writing it down either as part of the MOU or elsewhere.
Costs and Budget. Costs could include fees for an external professional evaluator, if used, as well as the planning costs – travel, time for meetings, phone calls, room rental, fees for licenses, printing, etc. Also consider equipment – laptops, projectors, copiers, etc. Getting agreement in writing about the expected costs and how they will be covered is critical to determining the scope of and commitment to evaluation.
What is the timeline for this MOU? We have traditionally used a single MOU specifically for the evaluation planning phase, and then created a new one for the implementation phase – but this is not the only option. You may wish to address the entire evaluation process in a single MOU. But it is very important to be clear about expectations on the length of time each phase takes. Some programs have moved relatively quickly through the evaluation planning phase (4 months), while others may take a year or more. We often begin by asking staff about their annual cycles – when are reports due? When is the next program being held? The answers to these questions will help determine a timeline that will work for the project.
At a minimum, the MOU is meant to be a vehicle to plan evaluation activities, and to help assure that all parties are clear about what is intended or expected, understand their individual roles as they pertain to evaluation, and agree on what can feasibly be accomplished in the timeframe allotted. This written agreement does not necessarily have to be a formal legal document, but it should be a consensus between all the involved parties, be put in writing, and made available for easy reference. A sample MOU can be found in Appendix I. Although this will ideally be accomplished through a series of face-to-face discussions, we have often worked through this step via teleconferencing and email.