Evaluation Framework

1. Introduction: How to Use These Resources
2. Getting Started: Planning for Evaluation
3. Theory: Explaining How Your Project Works
4. Outcomes: Goals and Objectives; and Their Relationship to Strategies
5. The Plan: Basic Components
6. Data Sources: Some Examples
7. Implementation: Putting the Evaluation to Work
8. The Report: Communicating the Results
9. Examples: Real Evaluation Plans
10. Resources: Index of Materials Supporting Evaluation

Professional Development Model

Background & Foundation

School Technology Needs Assessment (STNA)

Background
The School Technology Needs Assessment (STNA, say "Stenna") was originally developed by SEIR*TEC at SERVE in collaboration with the North Carolina Department of Public Instruction's Educational Technology Division, as part of the LANCET project (Looking at North Carolina Educational Technology).

The STNA was created to help building-level planners collect and analyze needs data related to implementation of the NC IMPACT technology integration model, as well as other contemporary frameworks for examining technology use in teaching and learning.

The STNA is typically accessed through a web address unique to each school, using a free online surveying system provided by SERVE. After a the survey has been completed by all staff members, it is closed and the address of a web-based report is provided to the individual responsible for coordinating the needs assessment. For schools preferring other options, a paper-and-pencil copy of the STNA may also be downloaded and freely reproduced for use, or items may be adapted to any other surveying method (e.g., telephone, email, or other web-based system).

A summary of the results of a study on STNA is available from SERVE (PDF).

A copy of a paper presented at the 2007 annual meeting of the American Evaluation Association in Baltimore, MD detailing the study is also available. Investigating the validity and reliability of the School Technology Needs Assessment (STNA).

A new version of STNA (3.0) was released in December 2006, although the older version is still available to schools wishing to compare newly-collected data with the results of previous STNA implementations.

Appropriate Uses
The STNA is intended to determine the collective needs of a school staff, related to the use of technology in education settings. It provides information to help school-level planners - administrators, technology and media specialists, and school or technology planning team members - make purchasing, resource allocation, or other decisions relating to technology. However, its most important use may be guiding building- and district-level decisions about professional development for educators, including those about content, timing, and type of opportunities provided.

It is not appropriate to use the STNA in any attempt to determine individual teachers' skills with, understandings of, or attitudes about technology; nor should this instrument be used to compare schools or their staffs.

Data and Reporting
The STNA collects perceptive data (what respondents think or feel) about a variety of broad areas of technology implementation in their school:

• The degree to which environmental factors support technology use
• Professional development opportunities offered and taken advantage of
• Uses of technology to support teaching and learning activities
• The impact of technology on teaching practice and student outcomes

Responses are analyzed through the online system and each report provides a picture of that school as a whole, presented as frequencies and percentages of responses to all items, and as bar chart representations of those values.

PDF guides to Interpreting STNA data are provided to help school planning teams make well-founded decisions about technology initiative implementations (STNA v.2.x, STNA v.3.0). These documents illustrate a range of sample item response profiles and describe how they might best be interpreted.

Using the STNA in Your School
A PDF step-by-step checklist for Implementing STNA in Your School is available to help guide deployment of STNA in your school. To use the online STNA, to arrange for initialization of your school's individual STNA. Be prepared to provide some specific information:

• The name of your school
• An accurate count of the number of respondents expected to complete the STNA
• The date on which staff members may start responding
• The date on which responses should end

Once the response period has ended, a report URL is provided only to the original school contact.

A sample of the online STNA is available for review. Note that, while this sample is completely functional and may be completed and submitted, responses will not be visible to SERVE or to the visitor reviewing the instrument.

Note that the current version of the STNA (3.0) includes items asking about the professional profile of respondents. This information is collected for the sole purpose of furthering ongoing study of the reliability and validity of the STNA instrument. In order to preserve and ensure respond confidentiality of individual responses, this demographic information is never made available to schools. It is not possible to identify individuals or their responses in online STNA reports.

The STNA inferences document (PDF) provides some guidance for interpreting and using your STNA results. View a presentation (in PPT or PDF formats) that describes STNA and provides general suggestions for interpreting the bar graphs provided in the online report.

You may also view a sample STNA online report, to see how results will be organized, once your school staff has completed the survey.

To deploy the paper-and-pencil STNA, go to the STNA Downloads page, choose the version you want, and freely distribute the PDF version of the document. Note that frequencies and percentages must be calculated manually for this version of the instrument.