2. Overview

 

The SEIR*TEC Framework for Formative Evaluation

The framework described here is primarily intended to guide formative evaluation of projects that apply technology to teaching and learning activities in classrooms. It is hoped that this framework will be valuable for a variety of reasons:

• The collective understanding of "how technology works" to improve teaching and learning has expanded, and old assumptions may no longer hold.
• Evaluation can be a relatively expensive endeavor and is generally underfunded as a component of education technology initiatives.
• Granting agencies are raising expectations of accountability that come with technology-focused awards.

Although internal, formative evaluation by a district or school cannot fully replace a comprehensive, professional evaluation, education programs can nonetheless benefit from formative feedback generated by those within the system.

The framework provides a step-by-step approach to help non-evaluators plan and implement efficient, well-founded, theoretically sound evaluations of technology projects in education settings by breaking a complex process into manageable pieces:

• Planning the evaluation
• Explaining how the project is supposed to work
• Establishing project goals, objectives, and strategies
• Developing the basic components of an evaluation plan
• Identifying data sources for the evaluation
• Implementing the evaluation effort
• Communicating the evaluation results

These pages also provide tested resources, data-collection instruments tailored to technology implementations, and examples to guide the above steps.

Evaluation Terms

It might seem simplistic, but the biggest challenge in developing a formative project evaluation plan might be assuring that everyone involved in the process is speaking the same language. The distinctions among terms can be critical to understanding and to building the consensus necessary to design and implement a feasible, accurate, useful evaluation. It is not necessary that all stakeholders agree on a universal definition of a term like "objective" but it is extremely important that they accept common word usages for the purposes of the evaluation effort.

One possible first step toward shared understanding is the SEIR*TEC Evaluation Term Matching Exercise (in PDF form). Using this exercise, have evaluation planning team members match evaluation terms with Thanksgiving Day metaphors, and then compare responses to the Matching Exercise Answer Key. Resolve differences in understanding by discussing the resulting matches. Finally, record definitions of these key terms, as they will be applied to your evaluation planning effort, and distribute them to all stakeholders.

Planning for Evaluation

The next step might be to establish an overview for planning by reviewing and discussing Guided Planning for Evaluation: An Overview (PDF). This presentation was originally prepared by SEIR*TEC for the Mississippi Department of Education's Evaluation Institute for EETT Grant Coordinators to...

• Provide an overview of the steps that grantees go through in developing an evaluation plan
• Provide an evaluation framework that will help evaluation planners make informed decisions about project implementation and impact
• Provide examples of evaluation plan components
• Offer worksheets for developing an effective evaluation plan

The process described in this presentation may also be supported by referring to How to Develop an Evaluation Plan (PDF) - a flowchart depicting a complete overview of the logical sequence that applies the materials available in these web pages.

Finally, a more comprehensive review is available in Planning into Practice. This publication is grounded in SEIR*TEC's ongoing work and details the process of creating a strategic technology plan. Chapter 7 examines development of evaluation plans tailored to technology implementations in teaching and learning, describes options for collecting data, and suggests methods that might be applied to analyze results.

The next major step is to precisely define the project that will be evaluated. Again, it is absolutely essential that all stakeholders agree about how the project works, how it will be implemented, and what successful outcomes "look like." Failure to work through this often difficult process will complicate a project evaluation - sometimes to the point of paralysis.

Next > 3. Theory: Explaining How Your Project Works

If you have questions or comments, contact

This page last updated 6/23/05