Guide to the Systems Evaluation Protocol
The Guide to the Systems Evaluation Protocol (SEP or “Protocol” for short) is more than just the sequence of steps and a list of factors to be considered when designing an evaluation. The Guide specifically articulates the unique facilitation techniques and strategies that the Evaluation Champion may use, as well as the role that he or she plays when conducting systems evaluation. Throughout this guide, we will refer to the reader as the “Evaluation Champion.” This term is intended to be inclusive, and applicable to any professional who may be using this guide to plan or help plan an evaluation. An Evaluation Champion should be thought of not only as a facilitator of the Systems Evaluation Protocol but as a driving force behind the creation of an evaluation culture within the participating organization. In addition to the Evaluation Champion, we will also refer to the “Working Group.” This is also intended to be an inclusive term, describing any members of the organization who are working together through the steps of the protocol. In some cases this may include collaborative program staff exclusively, while in other cases this term may refer to members of the organization from various levels in the organizational hierarchy (program staff, administrators, funders) as well as participants and related stakeholders.
The process of working through the protocol will consist of collaborative meetings that will seemingly spiral through several focal points over time, as well as ongoing work around building a culture of evaluation in the participating organization. This process is essential to the nature of the SEP. It is through these discussions that the organization and its programs will develop a new outlook on their work that will change both their understanding of how the program stakeholders perceive the program and their sense of purpose in what they are doing and why. The SEP is a standardized protocol that nevertheless enables any program to develop an evaluation uniquely tailored to that program. In this sense it addresses the administrative need in an evaluation environment to standardize evaluation approaches while respecting the variety of contexts within which programming is conducted.
Putting evaluation concepts into a simple set of steps which we call the Systems Evaluation Protocol requires that we present the Guide in a linear format. In fact, an important objective for us in this work has been to instill the idea that effective modern evaluation requires evaluators to move beyond a linear mindset. Good evaluation requires feedback, and is embedded within a dynamic changing system. Although any written document is by definition linear, systems evaluation is a non-linear and iterative process. We expect that in various contexts it will be appropriate to perform steps out of the presented sequence or in tandem, as well as to revisit steps repeatedly throughout the process.