8. Improve the School Climate and Evaluate Its Impact on
Teacher and Student Morale and Student Achievement

This kind of project requires two kinds of evaluation:

1. A formative evaluation to determine if the program to improve school climate has been implemented.

2. A summative evaluation to determine if the school-climate program has resulted in better teacher and student morale and improved student achievement scores.

I  Formative Evaluation

The task here is to determine if the school-morale program has been implemented. This is not something that can be taken for granted. You will need to assemble credible, objective evidence (not simply your own judgments) that the key elements of the school-climate program have been implemented. Thus, you will need to assemble credible, objective evidence (not simply your own judgments) on the following types of questions:

1. Does the school climate program exist?
2. Are its components in place, and are its activities occurring?
3. Is the program operating as it is supposed to?
4. If not, what changes are needed to make it operational?
5. Are funds being appropriately spent?
6. If they are needed, have competent staff been hired and trained?
7. Have any required books, materials, and equipment been purchased and made available to the teachers and students?
8. Is the program serving the intended students?
9. Are they receiving the intended educational information and services?

It is essential to establish that a meaningful school-climate program has been implemented before undertaking a summative, impact, or outcome evaluation of its effects. It makes no sense to evaluate the effectiveness of a program that does not exist-even though we regrettably see many instances of this.

II  Summative Evaluation

The two basic questions here are:

>Has there been an improvement in staff and student morale and an improvement on some measure of student achievement?

>Can any changes that have occurred be confidently attributed to the school-climate program rather than other factors?

There are two preferred evaluation designs for answering these questions:

(1) Administer pre- and post- achievement tests and collect staff and student morale data through surveys and interviews on both a group of teachers and students in a school where the school-climate program was implemented and a comparable control group of teachers and students in a school where it wasn't. If the improvements in morale and the gains in achievement in the school-climate-program school are statistically and educationally greater than those of the control group, you can confidently conclude that these gains were the result of the school-climate program. Simply administering pre- and post-tests to the treatment or program group without a control group will not do. Achievement scores and morale may have improved because of other conditions or factors besides the school-climate program. For comparison, you need some measure of change under the conditions of non-treatment. This is provided by the control group. (See the section on An Example of the Most Common Pitfall in Evaluating Education Programs in the Short Course on Evaluation Basics.)

(2) As an alternative, if a control group is not available, you can use what is known as the interrupted time-series design. If test scores and staff and student morale data are available for several years or periods prior to the introduction of the new program, and there is a marked improvement in these scores immediately following the implementation of the school-climate program that is sustained in following years or periods, you can confidently conclude that the improvements were the result of the new program.

For more information on these two designs, see the section on Alternative Summative Evaluation Methods in the Short Course on Evaluation Basics, and references 1 (Campbell), 4 (Cook & Campbell), and 8 (Rossi & Freeman) in the Evaluation References.

Design Selections | Evaluation Support Home
Applied Research Center | EdD Major | Ed Leaders Home/ Basic Statistics