evidence of humanity in a bureaucracy

(Phoenix, AZ) I’m here for the final stages of administering a federal educational test. That’s an immensely complicated business, with more steps and people involved than I would ever have imagined. To mention just one glimpse into the whole operation: students complete short essays that must be scored by human beings. The scorers must be hired (which requires giving them standardized tests) and then trained. The training requires a detailed scoring guide for each test item, with many examples of real students’ work. The training also requires trainers, who must be selected and trained. The trainers of the trainers, in turn, must be selected and trained. They all need guides and materials. As each group does its work, their performance is monitored by computers, and discrepancies are identified and rectified.

The guiding principles are consistency, standardization, reliability, and transparency. This is all very Weberian–it is a highly refined bureaucracy. And so it must be: test-takers and the public deserve consistency and transparency, and therefore everything must be recorded, disclosed, standardized, and tracked. Fittingly, we meet in a windowless room off a highway in suburban Arizona, surrounded by vast banks of computers. (More than 1 million individual essays will soon be scored at this center.)

The contemporary philosopher J├╝rgen Habermas distinguishes “system” from “lifeworld.” The system must be organized and structured along Weberian principles. The lifeworld is authentic and human, but disorganized. As my colleagues and I review real samples of student work for the purpose of creating general “system” policies, the lifeworld emerges. The scanned copies of handwritten essays contain idiosyncratic outbursts, cute misunderstandings, and wild misspellings–evidence of life that it’s our job to codify.

Habermas argues that system and lifeworld should be mediated by the “public sphere,” by open and fair discussion of values. The design and implementation of federal tests involves countless value-judgments–for example, whether and how to define students’ race. We on the federal advisory panel discuss such issues, but we have limited discretion, because legitimate decision-making power lies with the Congress and the public, not with any so-called experts. To the greatest extent possible, I would like to see educational standards, assessments, and statistics be topics of public inquiry and debate. None of it is secret, but the public debate is frustrated by an excessive deference to experts, superficial media, and a narrow focus on a few hot-button issues.