"Bad Teachers"

Please insert standard apology for not blogging in a while here.

To return to my interrupted stream of posts, I assume everyone has heard that Michelle Rhee has gotten herself into a spot of trouble. USA Today is reporting that one of Rhee’s “star” programs is most likely guilty to cheating to improve their standardized test scores. This is precisely the problem I wrote about a few weeks ago: standardized testing as a diagnostic tool is great, but standardized testing as a means of punishing particular districts, schools, or (most especially) individual teachers is a terrible, awful, no-good idea. Gosh, no one could have predicted…wait, I predicted it! I mean, how about this for a conundrum: when you design a high-stakes testing system, who is going to implement the tests? The school districts, and most frequently the teachers. All right, so who gets in trouble if the test scores are below expectations? Why, the teachers. Congratulations and welcome to your gigantic conflict of interest. Under the proposed regime, everyone in the educational system has an incentive to cheat – superintendents, principals, teachers, everybody. There’s a word for this sort of thing on the internets that I think is a good fit: epic fail.

Today’s critique of Matt Yglesias’ latest post on education: in response to criticism of Rhee and the use of high stakes testing, Matt points out that a large proportion of students perform very poorly on standardized tests. I believe this is what his philosophy professors would have called a non sequitur. He assuming the premise that high stakes testing is a means to improving educational performance – which is the very thing that critics of neoliberal education reform are contesting.

Oh, and do remember the nice things I said about the KIPP charter schools a few weeks ago? I might have to take it back - it turns out that they get more money and have higher turnover among students than regular public schools. Hence, their superior performance is called into question. (h/t Oliver Willis)

And now at last I shall return to the topic I promised to discuss two weeks ago: the focus on bad teachers. One of the latest fads of neoliberals is to focus on the problem of eliminating bad teachers from the classroom. The logic goes something like this:

1) research suggests that teacher quality is a key ingredient in student performance

2) given that some teachers are much worse than others, if we can get rid of weaker teachers then student performance will improve

2) collective bargaining agreements guarantee due process, making it difficult to eliminate teachers

3) therefore, by eliminating collective bargaining and making it easier to fire teachers, you can increase student performance.

Now there is definitely something to point #1. Teachers certainly have an effect on student performance. Eric Hanushek, one of the leaders in this field, has a good review here. Now where Hanushek gets really aggressive is with the idea that if you replace “bad” teachers with “good” ones you will necessarily improve student achievement. Well, maybe, assuming you have a good mechanism for identifying good teachers in a fair way. Which as I suggested before, we don’t. I’d like to take this opportunity to throw in one more piece of evidence that the teacher accountability systems that use the VAM are questionable: Rothstein’s recent article in which he uses future teacher effects to predict how students did in the past. Sounds crazy right? Well it is – that’s what Rothstein is tested, whether the VAM results were taking into account student sorting by teacher. I’ll let you guess what he found.

What we also don’t have is any research that tests what happens when we put this policy into practice. I for one am profoundly worried at the potential spillover effects on the rest of the teaching force at the focus on firing teachers rather than improving them, with respect to morale, retention, and recruitment. Furthermore, I have yet to see any evidence that the spillover effect from bad teachers is worse than the spillover effect from less effective employees in other professions. It’s also a real question whether the effects of a bad teacher in one grade are permanent, or whether future mediocre or good teachers would dilute/nullify their effects. As you can see there’s a ton we don’t know and haven’t studied carefully. Now we do have one field test of what happens when teachers don’t have due process protections: they’re called charter schools, and the result has been higher teacher turnover and efforts at unionization.

So what we have is a neoliberal proposal, championed by political conservatives, that might have a theoretical effect on the academic performance of some students, but for which there is no clear basis in the research literature and has never been adequately tested. Again.

Next time: teacher tenure.

Passing Of An Era

I was going to write more about education policy today, but news of the death of Elizabeth Taylor stopped me. I remember the first time I saw her in a movie: it was Cleopatra I think, a silly movie for which I have a soft spot, and one that played a big role in my love old movies. I don't know if I've ever seen another actress with the same strange mix of pluck, humor, bitchiness and breathtaking beauty. I only saw Who's Afraid of Virginia Woolf recently, but no one who saw her performance could ever question her acting talents. Rest in peace.

Evaluating Teachers

One of the central planks in the neoliberal agenda is what they call "accountability" - using standardized tests as an instrument for school improvement. There are a lot of pieces of the accountability movement, but today I'd like to focus on one that has gotten a lot of attention recently: using standardized test scores to making hiring, firing, and promotion decisions for individual teachers.

The other day Academic Cog linked to a NYT article showing a very scary looking equation used to evaluate teacher performance. The piece described pretty well how standardized tests scores are being used to decide who does and doesn't get tenure. Without getting too deeply into the methodological weeds, the basic approach is to try to isolate the effects that teachers have on student performance, controlling for other factors. The "value-added model" (or VAM) uses changes in student performance over time to rank teachers. These statistical estimates have been used to create unnervingly specific ratings for teachers, ratings that have been published in major newspapers. It's not often that social science statistics inspire such heated controversy (with major spreads in NYT (above), USA Today, and the Columbia Journalism Review), but that's what happen when statistics determine the fate of millions of teachers.

So what's wrong with the idea of using objective measures of student achievement to assess teacher performance? Well, as it turns out, quite a lot. Two pretty good reviews of problems with the VAM were published last year, one by Sean Corcoran for Annenberg and the other by Baker et al for the Economic Policy Institute. To summarize:

1. Imprecision: the statistical models have too much built-in error to make reliable determinations of individual teacher performance. A report by NCES suggests that there is at least a 25% chance of misdiagnosis (they suggest school-level analysis instead).

2. Instability: teacher and student performance tends to bounce around from year to year. An teacher who is considered very good in one year might be labeled as terrible in another.

3. Poorly constructed assessments: state standardized tests have all sorts of problems, such as being predictable (and thus easy to game). There's also some reason to think that the choice of assessment has a big effect on how a teacher performs. A major problem is that some tests have ceilings that are too low. If you're teaching a class in which 75% of your students score in the 90th percentile, are you a bad teacher because they haven't gotten better by the end of the year?

4. Model specification problems: it is very difficult to make sure that you have appropriately controlled for all the various confounding variables. As complex as Academic Cog's equation looked, the biggest concern might be too few variables.

5. Narrow subjects: there's an unfounded assumption that standardized tests accurately measure the whole of student knowledge in a given subject. In fact what they do is test how well students are able to answer questions that are amenable to standardized testing. Lots of things are hard to ask about in a standardized format. To make matters worse, even if we choose to believe that the assessments that have been created are good enough to be used to evaluate students accurately, what do we do for teachers who aren't teaching math or reading in grades 3-8? Are we just going to eliminate the other subjects? Or are only elementary and middle school teachers going to be on the hook?

6. Perverse incentives: a whole laundry list, not least of which is that teachers and principals are going to have every reason in the world to do nothing but testing drills all year, not to mention outright cheating.

Good statistical models for assessing student and teacher performance are very desirable, but what worries me most is that they are being used for purposes for which they were never intended. They're designed to determine general, aggregate influences, not label a specific teacher "good" or "bad." Policymakers and neoliberal activists are mis-using these tools. The research on using data to improve student performance states clearly that standardized tests should just be one of a range of assessment tools - not the final word. Value-added models aren't being used to identify what effective teaching practices, rather they are being used to identify scapegoats, to lay blame on individual "bad" teacher and to fire them. More on the whole "bad teacher" obsession of neoliberals next time.

Charter Schools

In honor of Leslie-MB, I'm going to start off with charter schools. Let me begin by saying that in principle I have nothing against charter schools. The idea of creating experimental laboratories of education reform that are under the loose supervision of public authorities is a good one. I especially like the original vision of charter schools, which would be organized by teachers eager to try out new means of teaching students. Seriously, let a thousand flowers bloom: modified curricula, team teaching, alternative assessments, differing class schedules, ability grouping, etc. etc. In the beginning charter schools looked like a wonderful opportunity to field-test ideas about improving eduction. If the charter school movement had stuck to this original conception, I would be moderately supportive.

Sadly, in practice charter schools have not worked out that way. Now we can't know for sure how innovative charter schools are because there isn't a lot of research on the subject. Which should tell us something! I mean, if the whole idea behind charters is creating new educational methods, why isn't there an avalanche of rigorous data? To the extent that we do know what's happening in charters, it appears that on the whole charters teach students in ways quite similar to traditional public schools. The chief "innovation" of most charter schools is to avoid having teachers unions. The most successful charters, like KIPP, have adopted a model that pushes its teachers extremely hard - we're talking 60+ hours a week hard. The results are predictable: high teacher turnover, periodic bouts of union activity, and a model that can't be scaled up. (To be fair, some evidence suggests that the higher turnover is due to the kind of teacher that is hired by charters rather than personnel policy, but this hardly makes things better)

There is a huge volume of literature analyzing the effects of charter schools, some of a fair amount of rigor and some not. Meta-analysis by CREDO studying the overall effects of charters suggests that charter school performance is very similar to traditional school performance. Some are good, some are bad, and most provide an education that is pretty much the same as that of traditional schools. This is consistent with my impression of the research literature: that on balance charter schools as a whole are no better than other schools, once background characteristics and the like are controlled for (that last is a really vital point).

Now on one level these ambiguous results shouldn't be a surprise. If charter schools are adopting innovative teaching methods, we should expect results that are roughly comparable to other schools. However, even if charter schools ARE being innovative, we STILL wouldn't expect their performance to be radically better than traditional schools for the simple reason that not all innovations succeed. After all, if improving education were easy the regular K-12 system would have done so ages ago. Finally, there's the possibility that a focus on innovation can actually reduce student achievement.

With such limited evidence that charter schools are superior to traditional schools, one wonders why there is such a powerful constituency in favor of them. I think it stems from several sources. There are people who believe that the regular K-12 system is ineffective through some mix of bureaucratic corruption, teacher incompetence, union malfeasance, overregulation, or what have you. There are also people who, influenced by neoliberal market models, have simply deduced that a choice-based system will inevitably lead to higher performance. There are wealthy foundations who are looking for a "big win" by identifying THE educational model. There are politicians who are looking for an easy political fix to education, something that makes it look like they are "doing something."

The problem with many charter school advocates is that they have confused a mechanism for identifying educational innovations with a direct instrument for improving education. Charter schools are a means, but its most passionate champions are treating it as an end.

To boil it down even more, let me say that if the charter school advocates were right, if introducing market mechanism in education and maximizing "administrator flexibility" were the secret to educational improvements, if charter schools were the panacea that its supporters often suggest, we would already know it. There are thousands of charters serving millions of students, and so far we just haven't seen the dramatic improvements in student performance that we've been promised.

None of this is to say that there aren't very good charter schools out there. There most certainly are. There are also very poor ones. As far as I'm concerned, any pundit or policymaker (or moviemaker) who breezily argues that charters are the cure for our nations' educational ills either don't know what they're talking about or pimping an agenda. Either way they should be treated with scorn.

A Lovely Example of Neoliberal Idiocy

As an aside, I would like to identify Matt Yglesias as one of the worst offenders of sloppy thinking on education by liberal-leaning pundits. I generally like Matt's blog, but he has a terrible habit of holding forth on subjects before he's done his homework. There must be something in the water in D.C. that makes liberal pundits become elitist hacks. The most recent example is his post on the inability of public schools to narrow the achievement gap. A recent paper suggested that the gap between white and black students manifests itself before they ever attend school and persists, largely unchanged, through there involvement in the K-12 system. Matt then proceeds to use this as an argument for cutting K-12 funding and putting it into early childhood education.

I had to stop myself from screaming when I read this post. Matt has completely missed the distinction between educational equity and educational quality. Are schools good at closing achievement gaps? No. Are they good at teaching students things? As a matter of fact they are, as evidenced by the greater knowledge possessed by 18 year old high school graduates as compared with 6 year olds. Even the worst school system teaches kids something. Matt is right that we could cut school funding without altering the gap between white and black students, but the performance of both groups would be lower. Matt's proposals might make white and black school achievement more equal, but they would also make both groups dumber.

I'm all for a greater investment in pre-k, but let's not be idiots about it.

The Pathologies of Education Reform, or Throwing Caution To The Winds

I've been vacillating on whether or not to write this post for years now, for reasons that aren't very difficult to guess. But whether I am becoming more rash in old age, or simply disgusted with the trajectory of public debate on this issue, I am going to set aside my caution and engage the topic directly. It may not matter either way, given the likely small number of readers of this blog. But whatever the consequences, I can't in good conscience remain silent.

For the last decade or more we have been undergoing a powerful wave of education reform, one that has enjoyed massive (and bipartisan) support among policymakers, thinktanks, foundations, and the press. It is characterized by high-stakes testing, accountability, and various forms of school choice. I've taken to calling this movement neo-liberal education, because it is a part of the broader move towards market-based reforms in the public sector and the deregulation of the market in the private sector.

Although I strongly agree that the educational sector needs reform, and that there are some elements of the proposed changes that are worthwhile, as a whole I think that neoliberal reform in education, as with like-minded reforms, is at best (to steal the expression of a friend of mine) a GWOT - a Giant Waste Of Time. Whether it is merit pay for teachers, the weakening of collective bargaining in the name of administrator flexibility, charter schools, "turn-around" school reform, or high-stakes testing, I think that as they are being implemented these reforms will NOT achieve their stated objectives. They will not close achievement gaps. They will not make the U.S. an international leader in math and science. They will not improve the overall quality of education for American children. The best case scenario is that in ten years we will realize that neoliberal reform was just another fad that failed to live up to its overzealous champions' ambitions, and we will move on to some other set of reforms.

That's the best case scenario. What about the worst?

Although neoliberal education reform will fail to achieve its overt goals, I think there is a good chance that it will accomplish the implicit goal of many of its principal proponents, which is to undermine the system of public education. Yes, we might still have something called a "public school system," but like so many other public services that have been contracted out, education will effectively be controlled by the private sector, for the benefit of administrators and shareholders, and to the advantage of our country's most privileged citizens.

In the coming weeks, I am going to lay out why I am taking such a strong stance on these issues. I will (as time permits) examine each of the components of neoliberal education reform and explain why I it is I think they are going to fail. Unlike so many pundits, I will try not rely on sloppy logic, gross generalizations, and misleading anecdotes. Instead, I will rely on the evidence, on the research in the field, which provides precious little support for the idea that neoliberal education reforms will accomplish anything that we might wish.

Principal Agent Problems and Progressive Taxation

I think I have one potential solution to the problem of prospective corruption I mentioned yesterday. Okay, this is a little wonkish, but bear with me.

Traditionally liberals have argued that richer Americans should pay a greater proportion of their income in taxes on a number of different grounds, including:

1) Taxes are necessary for a civilized society and they have a greater ability to pay
2) The wealthy benefit disproportionately from the existing socioeconomic order
3) Poorer and middle class citizens have a greater marginality propensity to consume, and therefore lower taxes on those groups will lead to greater economic growth. Conversely, concentration of wealth at the top leads to asset bubbles.
4) Economic arrangements should be to the benefit of the least advantaged member of society.
5) Capitalism has a natural tendency to concentrate wealth and progressive taxes are need to fairly distribute the social surplus.
6) Excessive concentrations of wealth can undermine democracy in the short term (because of the rich's greater ability to influence the political process) and the long term (because of the creation of hereditary oligarchies and the related negative effects of a declining middle class)

It's not an exhaustive list, and the categories aren't sharply defined, but you get the idea. So what I've been thinking is that there is a seventh argument, namely that steeply graduated tax rates can help address principal-agent problems. Principal-agent problems are one in which there is a difference in the incentives between those who "hire" others to perform a task (the principals, and those who are the ones who actually perform the task (the agents). Economists are quite familiar with principal-agent problems in the private sector, where CEO's have an incentive to pump up stock in the short term without any concern for the long-term good of the shareholder's company. In politics the principal-agent problem manifests itself as the difference between the policy preferences (ideal or express) of the voters and the career interests of elected officials.

Got it? Great. So here's my (very tentative) idea: if we heavily tax incomes above a certain amount, then there won't be a big pot of gold waiting for congresspersons once the leave office. Right now they have a reasonable prospect of becoming millionaires or deca-millionaires. But if there were a 70%+ marginal tax rate on the top incomes - which we had until 1980 - AND there weren't a million loopholes, then suddenly selling one's soul doesn't look so profitable.

Now I'm not talking about a 70% tax rate on people making $100K, or even $370 K, which are no at the 28% and 35% rate respectively. (Did you know that making 350K puts you in the top 1% of households? I didn't.) No, I'm talking about the top .01% or so of income earners, 140 thousand people who right now make 2 million a year and earn 10% of all the adjusted gross income in the United States (see the tax foundation if you don't believe me).

It would only be a start, unfortunately, since the average net worth of members of Congress is over six million. We should do something about that too.

Rethinking Political Corruption

Something that's been bothering me lately is the career trajectories of former legislators. Now I'm not talking about political staffers who serve for a few years and then go work for the companies that lobbied them. That's a substantial problem, but at the end of the day elected officials are responsible for their own actions. No, what I'm more interested in is the phenomena where members of Congress retire or leave office and then begin working as lobbyists or on corporate boards. Our political system is predicated on the notion that officeholders wish more than anything else to remain officeholders. The representative system simply presumes that by threatening to withhold their vote citizens can hold elected officials accountable.

My question is whether this presumption reflects reality. I haven't done the empirical research, but it appears that a lot of elected officials are receiving large financial rewards for their behavior while in the legislature. There's a two-year ban on lobbying after you leave office, but there's nothing to stop a corporation (it's always a corporation) from paying an ex-congressman at $1000000 a year salary for life as a reward for "good behavior" while serving in office. Here's an example: let's say a member of Congress gets an amendment through worth a billion dollars a year over ten years to a given company, for a total of 10 billion in gains for the company. The member of congress is defeated or retires and is paid that $1 million a year salary for the next thirty years, for a total of $30 million. A simple cost-benefit ratio suggests that the company in question will have enjoyed a 333 to 1 return on their investment. Not too shabby.

There's a lot wrong with this back-of-the-envelope calculation. A single member of congress isn't usually well placed enough, for example. But my worry is whether there are enough members of congress who in the back of their minds say to themselves "I'll vote the way the financial industry wants me to, even if I'm not sure it's the right thing to do, because they'll help me out in the election. And even if I lose, they'll take care of me later."

My question is this: what in the world could be done to stop something like this from happening? Under the new campaign finance regime, what's to stop corporations from sponsoring candidates for office with the understanding that the candidate will "play ball" while in office and be richly rewarded should they lose? What laws are there on the books to prosecute an elected official, when we're not really talking about an explicit quid pro quo?

The answer is.....nothing. Nothing at all.