| Activities |
active pieces of the program that reach program participants |
|
| Analysis |
the process of deriving order and meaning from data |
|
| Assumptions |
beliefs and thought patterns about how and why a program is expected to succeed, or that would prevent a program from succeeding which are not otherwise explicitly stated |
|
| Bar chart |
also called a histogram, chart with rectangular bars with lengths proportional to the values that they represent, such as that used for a frequency distribution. |
|
| Bias |
a constant error; any systematic influence – on measures or on statistical results — irrelevant to the purpose of the evaluation |
JCSEE (2003) |
| Bivariate |
Refering to two variables (as opposed to uni-variate or multi-variate) |
|
| Boundary analysis |
the process of defining the structure and elements of a program; specifically what is considered part of the program as opposed to activities or elements that may be outside the program |
|
| Box plot |
a graphical representation of the five number summaries of a data set; which include: the minimum, lower quartile, median, upper quartile and maximum |
Wikipedia |
| Case study |
an intensive, detailed description and analysis of a single project, program, or instructional material in the context of its environment |
JCSEE (1994) |
| Categorical variable |
variables assigned to some label or category and therefore assessed on a nominal scale |
|
| Chi-square test |
statistical calculation used to test how well the distribution of a set of observed data matches a theoretical probability distribution |
|
| Cleaning (data) |
after data collection but before transforming or analyzing data evaluatiors must screen the data for accuracy – allowing you to go back and clarify problems or errors due to incomplete, missed or illegible answers. |
Trochim (2006) |
| Coding (data) |
Coding is a process for both categorizing qualitative data and for describing the implications and details of these categories. Initially one does open coding, considering the data in minute detail while developing some initial categories. Later, one moves to more selective coding where one systematically codes with respect to a core concept. |
Trochim (2006) |
| Comparison group |
a sample or population who are comparable to the evaluation sample, but who participated in the program in a different way or did not participate in the program |
|
| Construct |
an abstract or general idea inferred or derived from empirical evidence |
|
| Continuous variable |
a variable that can take on any of a range of values |
|
| Control group |
a group as closely as possible equivalent to the treatment group but does not receive the treatment |
|
| Correlation |
the degree to which two or more sets of measurements vary together; e.g., a positive correlation exists when high values on one scale are associated with high values on another; a negative correlation exists when high values on one scale are associated with low values on another |
JCSEE (2003) |
| Correlation coefficient |
a single number that describes the degree of relationship between two variables, that will always be between -1.0 and +1.0 |
Trochim (2006) |
| Cross tab |
the process of creating a table to describe the frequency and distribution of variables |
|
| Cyberinfrastructure |
an interactive website |
|
| Data |
material gathered during the course of an evaluation which serves as the basis for information, discussion, and inference |
JCSEE (1994) |
| Data management |
the ongoing process of recording, documenting, tracking, protecting, and organizing data |
|
| Descriptive statistics |
statistics used to describe the main features of a collection of data in quantitative terms (rather than to support or disprove a theory) |
Wikipedia |
| Design (evaluation) |
an evaluation design structures the research, to show how all of the major parts of the research project — the samples or groups, measures, treatments or programs, and methods of assignment — work together to try to address the central research questions |
Trochim (2006) |
| Effectiveness |
the ability to produce an effect |
Wikipedia |
| Efficacy |
the capacity for beneficial change |
Wikipedia |
| Ethnography |
a research strategy used for gathering empirical data on groups/cultures. Data collection is often done through participant observation, interviews, questionnaires, etc. Ethnography aims to describe the nature of those who are studied through writing. |
Wikipedia |
| Evaluation capacity |
the potential to plan, implement and utilize effective, useful, and professional evaluation practice |
|
| Evaluation capacity building |
involves the design and implementation of teaching and learning strategies to help individuals, groups, and organizations learn about what constitutes effective, useful, and professional evaluation practice |
Preskill & Boyle, 2008 |
| Evaluation cycle |
the iterative process of planning, implementing, and utilizing an evaluation |
|
| Evaluation partnership |
building evaluation capacity through a relationship between the evaluation facilitator and the partnering program, organization or system |
|
| Evaluation Champion |
serves as the evaluation expert in the evaluation partnership, allowing the program partner(s) to remain the program expert(s) |
|
| Evaluation plan |
a guide for the implementation of an evaluation |
|
| Evaluation purpose statement |
the introduction to the evaluation plan, which summarizes the context, both current and historical, in which the current evaluation exists, the key assumptions of the program and its evaluation, the current evaluation questions and methods, and the evaluation questions that will be addressed in the future |
|
| Evaluation questions |
the broad questions about the program that the evaluation seeks to address |
|
| Evolutionary systems perspective |
programs, as well as evaluation, are constantly evolving as a result of interacting with their complex, interconnected environments |
|
| Facilitation (of evaluation) |
the process of partnering with and training program, organization or system staff, as an evaluation expert |
|
| Frequency |
the number of occurrences of a repeating event per unit time |
Wikipedia |
| Histogram |
a statistical graph of a frequency distribution in which vertical rectangles of different heights are proportionate to corresponding frequencies |
|
| Implementation |
the process of carrying out an evaluation plan; including activities such as data collection, management and analysis |
|
| Interview |
a series of orally-delivered questions designed to elicit responses concerning attitudes, information, interests, knowledge, and opinions |
Wheeler, et. al. (1992) |
| Program lifecycle |
the individual course a program takes as it evolves, changes, or remains the same, over time |
|
| Lifecycle analysis |
the process of identifying and describing a program’s current lifecycle phase |
|
| Logic Model |
an outline of a program's inputs, activities, outputs, outcomes, context and assumptions |
|
| Long-term outcomes |
the ultimate impact logically connected to earlier term outcomes in a logic or pathway model, likely to include broader social, economic, or environmental effects |
|
| Maximum |
the largest value in a data set |
Wikipedia |
| Mean |
a measure of central tendency calculated by dividing the sum of all the values by the number of the values |
JCSEE (1994) |
| Measure |
a method or tool used to collect information |
|
| Measurement |
the process of using a method or tool to collect information |
|
| Medium-term outcomes |
describe effects on participants that logically connect short-term outcomes to long-term outcomes |
|
| Minimum |
the smallest value in a data set |
Wikipedia |
| Mixed methods |
using both qualitative and quantitative evaluation strategies to address the same evaluation question |
|
| Most significant change approach |
the method of collecting data only in cases where the desired effect was most pronounced |
|
| MOU |
stands for memorandum of understanding, which is the written agreement between the participating program, its parent organization, and the Evaluation Champion |
|
| Objective |
a characteristic of an evaluation that minimizes the impact of bias and subjectivity |
Wheeler, et. al. (1992) |
| Organization |
an office or institution that consists of one or more programs |
|
| Organizational change |
any action or set ofactions resulting in a shift in direction or processthat affects the way an organization works. |
HSRDS (2000) |
| Outputs |
the by-products of activities that may serve as evidence that the activity was completed, but not part of the larger flow of logic which explains the effects on participants |
|
| Pathway model |
a graphical representation of the activities and outcomes that make up a program, and how they are interrelated |
|
| Post-only |
an evaluation design in which the observation or measurement takes place exclusively after the program |
|
| Pre-post |
an evaluation design in which observations/measurements take place both before and after the program |
|
| Program |
a series of activities conducted with the intention of producing some effect (outcomes) on participants |
|
| Program boundary |
an imagined "line" between the activities, outputs and outcomes considered part of a program and those outside a program |
|
| Program description |
a summary of the basic components and characteristics of a program |
|
| Protocol (evaluation) |
a step-by-step guide |
|
| Qualitative analysis |
the process of deriving order and meaning from data using non-numerical methods |
|
| Quantitative analysis |
the process of deriving order and meaning from data using numerical representations and statistical methods |
|
| Quasi-Experiment |
a study in which the subjects to be observed (sample) are not randomly assigned to different groups, but grouped according to a characteristic that they already possess |
Wikipedia |
| Random assignment |
units in the sample are randomly assigned to different groups or treatments in the study |
Trochim (2006) |
| Regression analysis |
a technique for analyzing the relationship between a dependent variable and one or more independent variables |
Wikipedia |
| Reliability |
the degree to which an evaluation consistently produces the same result |
|
| Reporting |
the process of communicating results and recommendations to internal or external stakeholders |
|
| Sample |
part of a population |
Wheeler, et. al. (1992) |
| Sample size |
the number of observations that constitute the sample |
Wikipedia |
| Scatter plot |
a type of mathematical diagram using Cartesian coordinates to display values for two variables for a set of data; used for graphically determining if there is an association between two variables |
Wikipedia |
| Scope (evaluation) |
how much of the pathway model the program staff intend to evaluate in a given evaluation cycle |
|
| Sensitizing concept |
a category or idea that the observer brings to the data as a guideline, but which must be examined in a particular context in order to be meaningful |
Patton (2002) |
| SEP |
stands for Systems Evaluation Protocol, which is a standardized protocol designed to enable any program to develop a uniquely tailored evaluation for that program |
|
| Short-term outcomes |
describe effects on program participants that are logically and directly connected with the activities |
|
| Statistically significant |
a result that is mathematically unlikely to have occurred by chance |
Wikipedia |
| Stakeholder analysis |
the process of identifying and describing the perspectives of all of the potential people and/or organizations that have a stake in the program and its evaluation |
|
| Stakeholders |
any person ligitimately involved in or affected by the evaluation . . . |
JCSEE (2003) |
| Standard Deviation |
the standard deviation is a calculated number that describes the extent to which scores are dispersed (spread out) from the mean, nearly all scores are typically within 3 standard deviations of the mean |
JCSEE (2003) |
| Statistical software |
computer programs specialized for statistical analysis |
Wikipedia |
| STEM |
science, technology, engineering and mathematics |
|
| Subjective |
a characteristic of an evaluation that yields results which cannot be empirically verified by another person |
Wheeler, et. al. (1992) |
| Subscale |
a smaller set of items on a measure that have shared validity and reliability independent of the larger measure |
|
| Survey |
a method for collecting quantifiable information about a population |
|
| System |
a parent organization to one or more offices or institutions, in which complex interactions between programs, organizations and their contexts take place |
|
| Systems evaluation |
an evaluation that takes into account the complex relationships between parts of the program and between the program and its context |
|
| Systems perspective |
taking into account the larger contextual and environmental factors around a program, organization, or system, including the complex interactions between each |
|
| Thematic content analysis |
a qualitative analysis process that systematically identifies themes, patterns and connections in narrative data |
|
| Timeline |
a calendar or list of dates showing the evaluation stages and activities, and indicating the dates by which they should be implemented and be completed |
Wheeler, et. al. (1992) |
| Treatment group |
a sample sub-group that is exposed to the program, project, or instructional material as well as all of the conditions of the investigation |
|
| T-test |
assesses whether the means of two groups are statistically different from each other |
Trochim (2006) |
| Univariate |
involving only one variable |
|
| Utilization (evaluation) |
the process of using an evaluation, and its results, to make decisions about program change |
|
| Validity |
the extent to which the test scores or responses measure the attribute(s) that they were designed to measure |
Wheeler, et. al. (1992) |
| Variable |
a characteristic that can take on different values |
JCSEE (1994) |
| Variance |
in statistics, describes how far values lie from the mean |
Wikipedia |