Coming Soon to a City Near You

First it was Los Angeles...then New York. Eventually Chicago will follow the Tribune's suggestion (see below) and release teacher evaluation information to the press. Your city might be next.

Gregory Michie, who teaches in the Department of Foundations and Social Policy at Concordia University Chicago, wrote Climate of disrespect for teachers gets worse for the Answer Sheet blog in the Washington Post. His article is a response to an editorial in the Chicago Tribune calling for a publication of teacher ratings like was done in New York and Los Angeles. Michie wrote,

Despite the evidence that value-added models are inconsistent, volatile, and inappropriate for assessing individual teachers, some districts and states (including Illinois) have still decided to adopt them as part of their teacher evaluation systems. It’s bad enough that they’re buying into an error-prone approach. But publicizing the results is beyond the pale. What possible purpose does that serve except to further disrespect and demoralize an already thoroughly bludgeoned teaching force?

The Tribune editorial, titled Grading Teachers: Illinois parents, demand this vital data, was published on March 3, not long after New York released the data for 18,000 teachers. What follows are my comments on the editorial. Read the editorial for yourself (I have not reproduced it all here). I have not linked my responses to references, however, I have included resources at the end which support my comments.

A Priceless Gift

March 3, 2012

New York City officials recently delivered a priceless gift to schoolchildren and their parents: They released ratings for about 18,000 public school teachers that showed which ones helped students excel academically and which ones didn't.

The editorial begins with a misconception as a basis for everything which follows. The assumption in this paragraph is that the ratings for the teachers show which of those teachers help students "excel academically." The ratings might show which ones get higher test scores, but even if that's true, it's not necessarily the same thing.

Wealth of Information

That wealth of information helps stakeholders understand the concept of "value added," and it is the gold standard for teacher accountability. Illinois needs to do this.

The "wealth of information" is invalid and unreliable. If this is the "gold standard" for teacher accountability then there's a serious problem. Good evaluation tools exist for teacher evaluations and I would encourage schools and districts around the country to find a system which works for them. Using student test scores to evaluate a teacher, even the so-called "Value-Added model" (VAM) is not a good measure. There are too many variables which the VAM doesn't address. There is too large a margin of error for the process to yield meaningful results. Standardized achievement tests were not designed to evaluate teachers, so they are not valid for that purpose. The large margin of error shows the unreliability of using the tests.

How it Works

Briefly, here's how the system works: If a student scores higher than 60 percent of his classmates in math one year, but higher than 70 percent the next, then that leap of 10 points is the "value" added by the math teacher. A teacher's rating is based not just on how high her students perform, but on whether they perform better, relative to other kids, than they have in the past.

Do students, their circumstances or their environment never change?

Here are some variables which might change from one year to the next:

Family: a death of someone, incarceration (or return), economic status, divorce, marriage, birth of another child, family illness.

Student: Personal illness (child was healthy during test week last year, but has a cold or other illness this year), mental health issues (as students grow things change, puberty, relationships), physical condition (maybe the child has a toothache this year, or they changed meds for ADHD or some other condition).

Classroom (other than the teacher): Student relationships with peers change, disruptive peers who might not have been in the class last year, loss of a friend who might have moved, condition of the building (a/c not working, not enough heat, noisy radiators, poor air quality).

Teachers are an important in-school variable for student success, but students and their environments are not identical from year to year.

Some True Facts

• The best-ranked teachers are scattered across the city, in schools large and small, wealthy and poor, The New York Times reported.
• There's a lot of teacher variation within the best schools. At one elite New York elementary school, just over half the teachers were above average, and the other half were average or below average.
• Teaching bright students truly doesn't guarantee a teacher a high ranking. Even if high-achieving kids performed well on tests, some teachers were rated below average because the kids were expected to perform even better. That is, the students didn't progress much.

These may all be true, however, the fact that the rankings are invalid and unreliable remains unchanged.

The Teachers Union

New York teachers union leaders fought hard to keep these so-called "value added" rankings from public scrutiny. They argue that the margin of error can be huge. That the scores rely on outdated information. And that standardized tests don't accurately measure a child's true grasp of a subject.

But a New York judge wisely ruled last year that such concerns, though legitimate, were outweighed by the potential benefits of releasing this information. The public "has an interest in the job performance of public employees, particularly in the field of education," Justice Cynthia Kern of the Manhattan state Supreme Court wrote.

I agree that the "public has an interest in the job performance of public employees" however it's interesting that no other public employees personnel ratings are being considered for public display. It's also important that if the public is to see the ratings they should be accurate. These are not. The judge, it seems, was not so wise after all.

The Rating System Relies on Test Scores

Value-added is not the only factor in judging a teacher's performance. A teacher's in-class abilities and mastery of curriculum also needs to count significantly.

But they don't. 40% of a teacher's rating should be based on the test scores, however, the other 60% of a teacher's rating is ignored if the 40% is below the requirement. The rating system says that if the teacher is not successful in the 40% based on the test scores, then they cannot get a satisfactory rating. In other words, the 40% counts for 100%.

But value-added ratings are a powerful indication of a teacher's effectiveness...

No, they're not.

Los Angeles, New York and now, Chicago -- Misinformation

New York is not the first big-city district to have its teachers rated. The Los Angeles Times crunched Los Angeles school district numbers and released about 11,500 teacher rankings in 2010. The newspaper's website was swamped by parents seeking that information. No surprise there.

It's not a surprise that parents are interested in the ratings. Nor is it surprising that so few parents understand the lack of validity and reliability of the ratings. The Chicago Tribune, instead of editorializing in favor of such a flawed evaluation system, should explain to their readers why standardized tests need to only be used for the purpose for which they were constructed, and why huge margins of error reduce the reliability of those tests.

Vital Data

Illinois parents need this same vital data. A sweeping state school reform law already requires districts to take student academic growth into account in a teacher's evaluation...

The data is inaccurate, not vital. The "sweeping state school reform law" is misguided if it includes ranking teachers by student test scores.

Merit Pay

A value-added system will help Illinois educators do a better job identifying and rewarding the most highly effective teachers with, we can hope, markedly higher salaries. The information also can help schools replace educators who simply aren't advancing their students' academic performance.

This is a simplistic call for merit pay, which we know doesn't work.

No teacher should be afraid of a fair evaluation. Teachers who are not performing should be given opportunities to improve or, failing that, should be removed from the classroom. The so-called Value-Added Model, however is not a fair way of evaluating teachers. There are too many variables over which teachers have no control. The VAM claims to eliminate these variables, but it doesn't. Evaluations based on student test scores, using the current VAM are invalid and unreliable.

Comment on my comments: It's interesting that teachers are being told what to teach and how to teach. Then, they are being blamed because the results are not what the "reformers" want.

Resources:

Poverty and Potential: Out-of-School Factors and School Success by David Berliner
Value-Added Evaluation Hurts Teaching: The harm behind the hype by Linda Darling-Hammond
Reporting of Teacher Performance by Walt Gardner
NY principal: Teacher scores inaccurate at my school by Elizabeth Phillips and Valerie Strauss
How to Demoralize Teachers by Diane Ravitch
NYC releases teachers’ value-added scores — unfortunately by Valerie Strauss

~~~