What’s in the Name?
The term “Netway” is derived from the phrase “networked pathway”. The system assumes that any program or project can be described with its own pathway model (a causal logic model), but these models are likely to be relatable — they are likely to share common activities, outputs or outcomes. The Netway is constructed so that when project educators and managers enter project information about activities, outputs and outcomes, the system can immediately identify and suggest other existing projects that have similar or common elements and enable the users to adopt or adapt these elements for their own projects while automatically creating networked linkages of their models with others. Each new project model adds to the online network of such models and can be accessed in turn by others. This also helps ensure that different parts of the system can learn from each other and that even projects with no direct contact with one another, can use the cyberinfrastructure to benefit from each other’s experiences.
Netway Development
The Netway was initiated in 2005 as an online tool in our work with 4-H at Cornell Cooperative Extension (CCE) in New York City. The task was to developing evaluation plans and implement evaluation for their outreach programs – the challenge was to make it easier. The Netway was created as a tool to help educators build program models and develop evaluation plans, and was initially intended to be a Participation and Activity Tracking System of evaluation data for the CCE system. However, out of that work we found a greater need for evaluation capacity building across the system, rather than a need for a system to evaluate program system-wide. Although system-wide evaluation was a goal, planning on how to use evaluation data prior to having the capacity to plan and conduct evaluations was putting the proverbial horse in front of the cart.
In October 2006 we received NSF funding (NSF Award #0535492) , “Building Evaluation System Capacity for STEM Programs”. Out of this work came the beginnings of the Systems Evaluation Protocol (SEP, or Protocol), and the Netway as a modeling tool that was used for online collaboration, developing program logic and pathway models, and drafting evaluation plans. Partnering with nine organizations, and 2-4 programs within each organization, each program worked toward developing a well-articulated evaluation plan that would address the program’s and stakeholders’ goals and objectives. We worked with one person from each organization (who we called the EPM – Evaluation Project Manager), who took the SEP and Netway back to their organizations for evaluation planning. Notable conclusions from this phase of project development included: a systems perspective has the potential to significantly enhance evaluation, practitioners began to view evaluation as an on-going process throughout the life of a program, and could provide key feedback useful for program development (not just end-of project evalution), and the different evaluation methods are suitable at different stages in a program’s lifecycle. We found that use of a formalized protocol was beneficial to our participants, and that the partnership with them – we as evaluation experts and them as the program experts – was effective in building good evaluation. And the Netway was a key tool that was both efficient and useful in their work.
The Systems Evaluation Protocol (referred to as either the SEP, or the Protocol) is a standardized ordered list of tasks that nevertheless enables any program to develop an evaluation uniquely tailored to that program. Putting evaluation concepts into this simple set of steps requires that we present it in a linear format. In fact, an important objective for us in this work has been to instill the idea that effective modern evaluation requires evaluators to move beyond a linear mindset. Good evaluation requires feedback, and is embedded within a dynamic changing system. Although any written document is by definition linear, systems evaluation is a non-linear and iterative process. We expect that in various contexts it will be appropriate to perform steps out of the presented sequence or in tandem, as well as to revisit steps repeatedly throughout the process.
In August 2008 we received additional NSF funding (NSF Award #0814364) to take this work and disseminate it. The SEP expanded from a set ot steps to include resources and documents used for building evaluation capacity. Documents and videos from our partnership presentations were developed into tools that were built into Netway – so that Netway, in essence, offered a self-directed option for evaluation. Netway tutorials and FAQs were added, along with the cyberinfrastructure for stakeholder mapping, associating evaluation measures with outcomes, improved interfaces for working with the models and evaluation plans, and areas to track implementation and utilization of evaluations. The SEP – including the Protocol step, the supporting theories, and the ECB resources – became better articulated and more fully developed into the Facilitator’s Guide to the Systems Evaluation Protocol. The SEP partnerships took on various formats and methods – in addition to training EPMS to take training back to their organizations we worked directly with the practitioners and staff, themselves, as well as launching our first “Virtual SEP” – where partners completed their evaluation planning on their own with nothing more than an orientation training from us. The SEP also expanded from simply Evaluation Planning, to include a protocol and support resources for evaluation implementation and utilization.
In June 2015 the Netway has expanded greatly beyond our original work with CCE 4-H programs – and is now used for various purposes. Although, in our mind, the resources are targetted towards supporting an evaluation system built upon a systems approach to evaluation, preliminary research into how Netway is used includes evaluation planning, program planning, team building, capacity building, project management. A restructuring of the underlying hierarchical structure of Netway will necessitate seasoned users to learn some new approaches to managing their programs in the Netway, but at the same time, this newest release will simplify collaborations between organizations, facilitate management of program reporting, and finally enable self-registration of new users.
Guidance and Help: