Berliner in Australia: The Testing Fiasco

MISUSE OF ISTEP

On December 15 I wrote that, instead of "pausing accountability" and waiting a year to use ISTEP to label teachers and schools, we ought to stop using it altogether because, there is no

proof that the ISTEP has been developed to include measuring the effectiveness of schools and teachers, in addition to measuring student achievement.

DAVID BERLINER

By coincidence, the following day, Diane Ravitch reported on a talk given by David C Berliner titled Teacher evaluation and standardised tests: A policy fiasco. You can read about the video presentation by Dr. Berliner at the Melbourne Graduate School of Education web site and watch the hour-long video below.

Berliner discusses the worthlessness of evaluating teacher competence and teacher training program effectiveness with standardized achievement tests. His impetus for writing the paper was the call by Arne Duncan for the evaluation of teacher training programs based on their students' students' test scores.

Berliner maintains that the tests are invalid measures of teacher quality. To use them to measure the quality of teacher training institutions and programs is even worse.

He lays the blame for low test scores on America's high child poverty levels..

Teachers and teacher preparation programs are perfect targets to take legislators minds off of all the poverty and inequality that make some of America's education systems an international embarrassment. Blaming teacher education programs and the teachers they produce for disappointing standardized achievement test scores appears to me to be a diversion, of the type used by successful magicians. Blaming institutions and individual teachers directs our gaze away from the inequality and poverty that actually gives rise to those scores. In the same way a magician can divert attention of an entire audience when they make a person or a rabbit disappear.

He lists 14 points which explain why using test scores to evaluate teachers and teacher training programs is invalid.

Effects of Poverty vs. Effects of Teachers

The first point is that "reformers" who insist on using standardized test scores for evaluation of teachers and teacher training programs confuse the effects of poverty with the effects that teachers have on their students.

When using standardized achievement tests as the basis for inferences about the quality of teachers and the institutions from which they came it is easy to confuse the effects of sociological variables on standardized test scores for the effects that teachers have on those test scores. [14:15 on the video]

Blaming Teachers is Inconsistent with our Moral Code

Point 2 – We don't hold pastors responsible when parishioners kill themselves or others. We don't hold parents responsible for the actions of their adult children.

The logic of holding schools of education responsible for student achievement does not fit into our system of law or into the moral code subscribed to by most western nations. [19:48]

Clients Don't Always Comply

Point 3 – Medical schools and dental schools aren't held to the same standard. Poverty will result in lower life expectancies, poorer dental health, and poorer health in general, yet we don't blame doctors for their patients' illnesses, or dentists for their patients' oral problems. We don't tell them that "poverty is no excuse" or "poverty isn't destiny." We don't close hospitals or dental offices which treat the poor and we don't blame health professionals who work with poor people.

Professionals are often held harmless for their lower success rates with clients who have observable difficulties in meeting the demands and the expectations of the professionals who attend to them. [20:09]

Berliner added,

No one is proposing Heal for America so recent college grads can spend two years in an inner city emergency room.

Competent Teaching can Occur Independent of Learning

Berliner fourth point is that practicing good medicine is the goal of medical care, even when diseases can't be cured. The same is true for teaching and learning.

People accept the fact that treatment in medicine may not result in the cure of a disease. Practicing good medicine is the goal, whether or not the patient gets better or lives! It is equally true that competent teaching can occur independent of learning, although this appears to be too difficult a concept for our Secretary of Education.[23:48] [emphasis added]

We don't have to use invalid measures just to evaluate teachers. Other methods are available. See, for example, Linda Darling-Hammond's Creating a Comprehensive System for Evaluating and Supporting Effective Teaching and PARS, from Montgomery County, MD (note: the latter is no longer in use because the legislature requires that tests be used to evaluate teachers!).
Berliner said,

There are other quite acceptable sources of data besides standardized achievement tests for judging the efficacy of teacher education programs and their graduates.

Is Teaching to the Test Good Instruction?

What is good instruction? Berliner says, in point 5, that there is a confusion about good instruction because of the reliance on standardized tests. Is success in raising test scores good instruction?

My government's reliance on standardized achievement test scores as the only acceptable source of data about teacher quality, will inevitably promote confusion between what we mean by successful instruction on tests, and what we mean by good instruction, about some values we hold about what teaching and learning should be like. [27:20]

Tests Are Not Sensitive to Teachers' Effects

The effect of teachers on student achievement tests is actually very small. Berliner's 6th point begins at 37:14 and continues for several minutes. He says,

Although teachers may have profound affects on individual students, they do not affect standardized achievement test scores much at all. [43:28]...Teachers are not affecting test scores very much, yet the test scores are used as ways to blame teachers, schools of education in New Orleans, Chicago, Los Angeles, Philadelphia and dozens of other cities.

Teachers' Effects Aren't Permanent

Not only is the teacher effect on student test scores small, but what there is of it doesn't last – his 7th point.

Teachers affects on standardized achievement test scores fade quickly. [43:50]

Student Achievement Tests Aren't Validated for Anything Other Than Student Achievement

I wish Dr. Berliner's paper was available to read. Because of lack of time, he rushed through points 8 through 14. Here they are...

8. Observational measures of teacher competency and achievement tests of teacher competency do not correlate well. [44:18]

9. Different standardized achievement tests, both purporting to measure reading or math or science at the same grade level, will give different estimates of teacher competency. [45:40]

10. The administration of standardized achievement tests at different times of the year, will yield different estimates of teacher effectiveness. [46:13]

11. No standardized achievement tests have provided proof that their items are instructionally sensitive. [46:32]

12.Teacher effects show up more dramatically on teacher made tests than on standardized achievement tests because the former are based on the enacted curriculum, while the latter are based on the desired curriculum. [47:46]

13.The opt-out testing movement invalidates inferences about teachers and schools that can be made from standardized achievement tests results. [49:56]

14.Assessing new teachers with standardized achievement tests is likely to yield many false negatives. [50:17]

He concludes...

Standardized achievement tests are remarkably insensitive to teacher effects. [51:24]

In other words, standardized achievement tests aren't changed much by classroom teachers. Other variables are more important, especially those outside of school.

[This explains why] Teach for America's new and grossly untrained teachers do not seem any worse on standardized achievement test given to poor children than do experienced teachers. The tests are simply too insensitive to instructional quality, while being highly reactive to the income, social class, quality of the neighborhood, and the home lives that are presented by the students of Teach for America instructors, as well as the better trained and more experienced instructors. [53:33]

STOP MISUSING TESTS

We're using standardized achievement tests incorrectly. They are invalid as a measure of teacher competence, school quality, and teacher training program effectiveness. The discussion of whether or not to use this year's ISTEP tests to evaluate teachers and schools is irrelevant. We shouldn't be using any standardized student achievement test to evaluate teachers or schools.

The incompetence comes from those who insist on and/or pass laws requiring schools, states, and state departments of education, to misuse already questionable measures of student achievement. That's not accountability. It's irresponsibility.

###