Author Archives: Peter Levine

the value-added debate in education policy

The debate about assessing teachers’ impact has reached full volume. The Los Angeles Times recently released a public database that rates teachers’ “value added,” the New York Times has a new front page article about that kind of method, the Economic Policy Institute has published an important paper by 10 famous authors against it (pdf), and various prominent bloggers have weighed in.

We must assess the performance of public employees whom we pay for important public tasks–teachers included. Everyone who has ever been inside a school knows that teachers differ in their skills, relevant knowledge, and motivation. Once upon a time, we trusted educators–teachers, administrators, and unions–to assess themselves, but there is pretty broad dissatisfaction with that approach today.

The leading solution–enshrined in federal and state law–is to use standardized test scores to assess teachers. But now we’re supposed to use them in a sophisticated way, not just looking at the average score for each class (which is evidently affected by many factors other than the teacher). The leading sophisticated approach is to assess average changes in a teacher’s students over time. In essence, that method controls for students’ starting position and relies on the Law of Large Numbers to even out random or external factors that might affect any given kid.

It’s not a crazy theory–it has some research support, especially from the groundbreaking work of William Sanders–but notice how many premises and causal relationships the full strategy assumes:

This can go wrong in so many ways. Tests can be poor measures of students’ competence: they are never perfect measures. The Law of Large Numbers does not apply in this case, because each teacher can have a significant impact on only a modest number of kids. Hence there are large random fluctuations in value-added scores.

I have never seen evidence that parents try to place their kids in schools with the highest “value-added” teaching staffs. It would be odd if they did, because a student benefits more from a privileged peer group or a good school climate for learning than from teachers who add the most to standardized tests. (Larger increases can be achieved in low-income schools that don’t face “ceiling effects,” but you don’t see affluent parents enrolling their kids in those schools to reward the teachers.)

When teachers use standardized test scores to modify their own performance, they often “teach to the test” and narrow the curriculum. When administrators use such data, they do not consistently enhance the strength of their teaching staffs; they certainly don’t make the workplace more desirable for talented teachers. Even if a school’s faculty does add more average value to test scores, that doesn’t mean that graduates will become better citizens–or even that students will stay in school.

Kevin Drum thinks we face a Hobson’s Choice: no tests and no accountability, or poor accountability through testing. “The criticisms of value-added seem compelling. At the same time, if a teacher scores poorly (or well) year after year, surely that tells us something? At some point, we either have to use this data or else give up on standardized testing completely.”

I’m not saying that the answer is easy, but there are alternatives to this dilemma. We could reorganize schools so that teachers were able to hold one another more accountable: what I have called “internal accountability.” (Evidence from other fields shows that when internal accountability system are replaced with external measures, people become less motivated to do good work.) We could also bring parents into schools as partners, not just consumers, and boost what I have called “relational accountability.”

Either way, we would shift the metaphor. Teachers wouldn’t be service-providers whose service must be measured in a standardized way. They would be members of a community (also comprised of families), who hold one another accountable for contributions to a common task.

These ideas may sound idealistic, but they actually make fewer assumptions and leaps of faith than the supposedly hard-nosed strategy shown in the diagram above–which is embodied in current law.

Jonathan Lethem, A Fortress of Solitude

I recently read The Fortress of Solitude, a 2003 novel by Jonathan Lethem (having previously read Motherless Brooklyn, a funnier and perhaps tighter book by the same author). Fortress of Solitude has been compared to Portrait of the Artist as a Young Man: both are heavily fictionalized memoirs that begin in early childhood, when language and memory are still unformed, but emotions are raw and potent. Both focus on a sexualized and delinquent adolescence, deal with questions of national or racial identity, explicitly consider art and aesthetic theory, and end with the protagonist as an author reflecting on his own story.

Joyce’s character lives in British-ruled Dublin late in the 1800s, whereas Lethem’s hero grows up as one of two white boys in an otherwise African American block and school in the Brooklyn of the 1970s. The local bullies, the protagonist’s best friend, and his main girlfriend are all Black, while in Portrait of the Artist the key figures are Irishmen. Joyce’s hero debates Shakespeare, whereas Lethem’s writes about soul and Motown.

I found some personal resonances. I’m just a couple of years younger than Lethem and his fictional protagonist. My aunt and uncle actually lived not far from his fictional setting. My father, a cousin, and several other people I’ve known attended the high school where the hero studies. My college was not much different from his. Black-White relations, graffiti, punk, and the condition of bankrupt New York City were peripheral or contextual issues for me, central in the plot of Fortress of Solitude.

It’s an ambitious or even risky book. The biggest risk is departing from a fully naturalistic plot: let’s just say that some things happen in the novel that could not happen in the real world. I felt it become somewhat slack in the middle, once the hero leaves Brooklyn, but become suddenly taut again at the end when all the plots collide. Like the plot, the prose is ambitious and risky. Consider, for instance, this early paragraph with its evocation of filtered childhood memories, its free indirect discourse (where do Isabel’s thoughts begin?), and the use of “ribbon” as a verb:

    The boy lingered in the study and paged through Isabel’s photo albums while the mother sat on the back terrace, smoking. Isabel watched a squirrel ribbon the telephone pole, begin to scurry across the fence top. The squirrel moved as an oscillating sequence of humps, tail and spine bunching in counterpoint. Some humped things are elegant, Isabel mused, thinking of her own shape.

I think Lethem pulls off a fine novel, although sometimes it’s a close-run thing.

what happened to the new Obama voters?

Project Vote is pushing an important line of argument. They say that our policy debate is distorted because the media is fascinated with the Tea Partiers (“Who are they? What do they want? Will they affect elections?”) and is ignoring the huge number of new voters who turned out in 2008. Those new voters tended to be younger, less wealthy, more racially diverse, and more politically progressive than the typical US electorate, and they won a national election. If the press today would constantly ask, “Who are they and what do they want?” the whole policy debate might be quite different.

Lorraine C. Minnite writes, “heading into the 2010 congressional midterm elections the views of traditionally under-represented groups who were mobilized in record proportions in 2008 have been drowned in tea.” See her “What Happened to Hope and Change? How Fascination with the ‘Tea Party’ Obscures the Significance of the 2008 Electorate” (PDF) and a soon-to-be released Project Vote survey.

Reporters focus relentlessly on predicting the next national election. (I’ve quoted the former CNN political director, Tom Hannon, saying, “the most basic question about [an] election … is who’s going to win.”) From that perspective, it’s somewhat rational to focus on the Tea Partiers and not the recent Obama voters. Current polls that screen for likelihood of voting in 2010 suggest that the electorate will shift rightward again in 2010 because of who turns out. Thus, if you want to predict the next election, it makes sense to focus on the new conservative voters. Two important caveats, however, will probably be missed. First, the Tea Party will not represent the median voter, who will be moderate; and second, the electorate will probably swing back leftward in 2012.

Assuming that the media (and the blogosphere) continue to focus on predicting the 2010 election, the only way to shift the discussion is for progressive constituencies to threaten to vote. They need to tell pollsters that they are excited to vote, and they need to take public steps–like marches and protests–that indicate mobilization. That’s how the game is played right now, and they’re not playing well.

But the game isn’t satisfactory. “The most basic question” about politics is not “who’s going to win.” The most basic question is: What should we do? Although the press can’t answer that for us, they could provide information relevant to our decisions.

From that perspective, “Who will win the next election?” shouldn’t matter much. At most, it should have a modest impact on our strategic plans, but it should not cause us to change our own goals. (Thus the relentless focus on the horse race is problematic.) Who voted in the last election is perhaps a bit more relevant, because the winners presumably have some democratic legitimacy as the current governing coalition. Who might vote if we changed our politics is more interesting, because it invites us to consider a wider range of strategies. I’ll be looking forward to the Project Vote survey for that final reason–it will suggest ideas about how we might be able to mobilize new progressive voters with new progressive policies.

Where Harvard Meets the Homeless

Scott Seider has published a new book entitled Shelter: Where Harvard Meets the Homeless. It’s about a homeless shelter that is entirely managed and staffed by Harvard students.

Most of our work at CIRCLE concerns the civic engagement of people far different from those young leaders. We focus on the half of the population that does not attend college at all, let alone highly selective, private, four-year universities. But Seider’s topic is an important one because the kinds of people who gravitate to ambitious civic or political organizations at institutions like Harvard will soon run strategically important parts of our civil society and politics.

This generation is certainly different than their predecessors who would have flocked to Students for a Democratic Society and tried to block Robert McNamara from leaving campus. Today’s Ivy League undergraduates are more entrepreneurial, probably better organized, possibly more thoughtful, but lacking a comprehensive theory of how to change society.

As I wrote in my blurb, “Scott Seider’s rich and insightful study of Harvard students who run a homeless shelter provides an informative portrait of today’s young leaders and their struggle to understand and confront injustice.”