How to Use These Resources
These materials are provided to help school and district staff members who are not trained evaluators build their organizations' internal capacity to evaluate projects that apply technology in education settings. It can be used in two ways:
You can start at the beginning of this list of materials and descriptions and follow the link at the bottom of each page to each successive section to work through the steps required of a quality project evaluation. Or, if you have a specific need for, or interest in, information about evaluation materials, you can either jump to an individual section of the SEIR*TEC formative evaluation framework, or use the index provided in the Resources section.
What is Evaluation?
Evaluation is the process of judging the value or worth of a product, project, or program. Although the SEIR*TEC materials focus on formative project evaluation - intended to help project managers understand how an effort is working, so they can make decisions about revising or improving implementation - many of the concepts presented can be extended to evaluations of other things (personnel, for example) or for other purposes.
Why is Evaluation Conducted?
Evaluation is typically conducted for three reasons: (1) to determine what type of project should be developed (often referred to broadly as needs assessment), (2) to determine if a project has been implemented as intended (implementation evaluation), and (3) to determine the effects of a project (impact evaluation). There are other terms recognized by evaluators as describing the same purposes but these three terms will be used here for the sake of consistency.
Needs Assessment - Sometimes, evaluation is conducted before a project is even started (Rossi & Freeman, 1989). This needs assessment (or conceptualization) phase involves gathering data to determine gaps between the current state of affairs in a particular situation (e.g., technology integration) and the desired or optimal state. Project managers and other stakeholders can then determine whether there is a gap and if so, which are the most appropriate and pressing project activities to foster and support.
To assist with defining school technology needs, educators might consider the School Technology Needs Assessment, (STNA, say "Stenna") developed through a collaboration between SEIR*TEC and the North Carolina Department of Public Instruction." Another option is "TAGLIT: A Tool for Measuring Project's Results" - an assessment tool that can be used to determine how technology is being used in a school.
Implementation Evaluation - Evaluation is often conducted to monitor the fidelity with which the activities of a project are being implemented. A project manager who is accountable to a funding agency may be interested in documenting internal processes and lessons learned, or a project sponsor might want to verify that funded projects are being implemented as promised. In such cases, evaluation can help to verify that the specified target populations are receiving the promised services, or the sponsored procedures are being carried out as indicated (i.e., "We said we would do X, and here is the evidence that we did X.").
Impact Evaluation - Evaluation is also conducted to determine if a project has met its goals or has achieved the desired outcomes. This type of evaluation is the most powerful to argue for project continuation, as it provides evidence that a particular initiative is related to, or influenced, a positive change (i.e., "We said the project would effect X, and here is the evidence that the project has effected X.").
When is Evaluation Conducted?
The work of evaluation can span the life span of a project from pre-conceptualization, to monitoring implemented initiatives, to judging any effects after implementation.