Evaluation is the process of judging the value or worth of some product, project, or program. The content in this Web site is focused primarily on project evaluation. The results of evaluation help stakeholders make decisions about revising/improving or continuing/ending projects (e.g., the procedure needs improving, the project is not worthy of continued funding).
[ Top ]
Evaluation is typically conducted for three reasons: (1) to determine what type of project should be developed (needs), (2) to determine if a project has been implemented as intended (accountability), and (3) to determine the effects of a project (utility) (Rossi & Freeman, 1989).
Some suggest that evaluation may be conducted before a project is ever conceptualized (Rossi & Freeman, 1989). This type of evaluation may be termed conceptualization or needs assessment, and involves gathering data to determine gaps between the current state of affairs in some situation (e.g., technology literacy) and the desired or optimal state. The evaluation defines for stakeholders whether or not there is a gap, and if so, which are the most appropriate and pressing projects to foster and support.
RESOURCE: To assist with defining school technology needs, educators should read, "TAGLIT: A Tool for Measuring Project's Results," which describes an assessment tool that can be used to determine how technology is being used in a school. As described in the article, TAGLIT has been used to identify certain needs in schools, such as the need for professional development and training.
Evaluation is also conducted to monitor the implementation of a project. A program manager who is accountable to a funding agency may be interested in documenting internal processes and lessons learned, or a program sponsor may wish to verify that funded projects are being implemented as promised. In such cases, evaluation can help to verify that the specified target populations are receiving the treatment as promised, or the specified procedures are being carried out as indicated (i.e., "We said we would do X, and here is the evidence that we did X.").
Finally, evaluation is conducted to determine if a project has met the goals set for it or has achieved desired outcomes. This type of evaluation is the most powerful to argue for project continuation, as it provides evidence that some initiative is related to or caused a positive change (i.e., "We said the project would lead to X, and here is the evidence that the project has effected X.").
[ Top ]
As noted above, the work of evaluation can span the life span of a project from pre-conceptualization, to monitoring implemented initiatives, to judging any effects after implementation. The terms formative and summative evaluation are often used to specify when an evaluation is conducted. A formative evaluation occurs when a project is being designed or implemented to determine if ongoing processes should be revised. A summative evaluation occurs after a project has been implemented to determine if the initiative met the goals set for it and therefore should be adopted or continued as an effective treatment. Summative evaluations take two forms--effectiveness studies just after a new project has been implemented in full, and impact studies several months to years after a project has been implemented in full (longitudinal).
[ Top ]
Evaluation is related to and often confused with assessment. While assessments are used to determine how much a student knows or how effectively a student can perform some task relative to a criterion or standard, evaluations are used to determine how well a project has achieved the criteria or goals established for it. If a project's goals include "improved student achievement," however, then the evaluation would likely utilize assessment data. So while assessment and evaluation are different, evaluations often utilize assessment data to help make decisions about the value of the project under study. Evaluation is the more inclusive term; it may use assessment, but assessment does not use it.
[ Top ]
Evaluation is related to research in that both utilize similar data collection methods and statistical analysis procedures. Evaluation differs from research in focus and involvement, however (Ohio Department of Mental Health, 2002). Research is based on existing theory and attempts to tease out the relationship between variables to add more knowledge to an existing field, while evaluation seeks to determine if a project has met the goals or criteria established for it. Adding to theory or developing a new theory is the focus of research, not evaluation. This is not to say evaluation conclusions have not been used to develop theory, just that the primary focus of evaluation is quality checking for the attainment of goals irrelevant of their contribution to theory.
Further, individuals or groups who have reviewed relevant literature, designed a project, and wish to study its effects, typically conduct research on their own intervention. An evaluator, however, is typically hired as an unbiased third party to estimate the value of some project that they probably had no part in designing.
[ Top ]
An evaluation plan represents your outline for the evaluation process. In a typical request for grant proposals (RFP), you will be asked to provide an evaluation plan. Given the large number of components that should go into an evaluation plan, the RFP page limitations for evaluation plans are often ridiculously low. Regardless, every plan that you prepare should address, either in depth or in short, each of the following components.
Note: a well-prepared evaluation plan can be readily converted to an evaluation report. An evaluation report should include all of the information provided in your original plan, as well as additional sections that outline results, a discussion of the results, recommendations, and limitations to the study (reliability, validity).
[ Top ]
RESOURCE: For more information, read "Tips for Writing an Evaluation Plan for a Technology Grant" by Elizabeth Byrom, SEIR*TEC Principal Investigator.
RESOURCE: For more information on and a general overview of the evaluation process, read about the three-stage process most often employed by school districts "Steps in Evaluating a School or District Technology Program" by Jeff Sun, Sun Associates.
Ohio Department of Mental Health. (2002). Program evaluation vs. research. [On-line]. Available: http://www.mh.state.oh.us/offices/oper/feature1.html
Rossi, P. H., & Freeman, H. E. (1989). Evaluation: A systematic approach. Newbury Park, CA: Sage Publications.
Forming an Evaluation Committee
Use the Evaluation Committee Composition Matrix as a worksheet when planning your evaluation committee.
[ Top ]
|For more information, contact|