Author Archives: Peter Levine

making tests

I’m at the Educational Testing Service in Princeton, NJ, helping them to cook up a test. There are many cooks at work on this particular broth. In fact, what strikes me most about the process of designing a national student assessment is the enormous complexity of the process. There are laws, policies, rules, budgets, curricular standards and objectives, “items” (i.e., questions of various types), answer keys, trainings and guides for scorers, data from preliminary laboratory tests of items, pilot tests of whole exams, statistical results from the pilot tests, revised items and instruments, final results, statistical scales, summary measures, and reports. There are content experts, item-writers, statisticians, psychometricians, scorers, trainers, trainers-of-trainers, and various layers of contractors and government agencies and reviews.

If you are prone to distrust or dislike pencil-and-paper exams (and I understand and respect those arguments), all this apparatus may seem like a bureaucratic and technocratic nightmare. Indeed, any test involves countless value-judgments, guesses, and compromises, often buried in technical or administrative jargon. There is something extremely “Weberian” about a government-sponsored exam or assessment. It is a classic example of the effort to standardize and measure in order to control and improve.

On the other hand, if you are not directly acquainted with the process of test-design at the federal level, you might not realize how many different people struggle to develop and implement tests that reflect ethical principles of fairness, reliability, relevance, and social significance. The result is, if nothing else, the product of a lot of hard work.

the administration’s civic engagement agenda

This is a very interesting report of a meeting convened by Beth Noveck, the new director of the White House open government initiative. Participants included many of the best scholars and practitioners in the field, and the discussion summary is available for all to see. Beth apparently began by saying, “We’re looking for ideas and recommendations on how to create a more transparent, participatory, and collaborative government.”

Meanwhile, the civilian service agenda continues to move forward. There was money for AmeriCorps in the stimulus package, and the GIVE Act (HR 1388) is moving in the House. [Update at 5:30 pm: it passed today.] This is the equivalent of the Senate’s Kennedy-Hatch Serve America Act. It would dramatically enhance the quality and quantity of service opportunities and would direct federally-funded service toward three major social objectives: carbon reduction, health care, and high school dropout prevention.

I see these as two important planks of a “civic platform.” I have spent considerable amounts of my own energies in support of transparency and online engagement, on one hand, and service, on the other. In fact, my first national summit on service was in 1988, when I was an undergraduate; and my first full-time job was at Common Cause, where I worked on disclosure issues.

But I think these two planks are inadequate on their own. They will enlist specific subcategories of Americans in civic engagement. Certain people will get very excited about commenting on federal policy in interactive websites; others will spend a year of their lives tutoring or building houses. The vast majority of citizens, however, will do neither. Besides, neither commenting/discussing nor direct service exhausts the types of work that active citizens should do. In the worst case, the active commentators online will develop ideas that no policymaker can enact, and the AmeriCorps volunteers will provide direct assistance without addressing deep social problems. Both groups may be discouraged.

Other important planks should include:

  • National (and also local) discussions of issues that involve recruited citizens who represent the population as a whole.
  • Training programs and conferences that help federal civil servants to collaborate day-to-day with community-based groups.
  • Changes in key federal policies such as the Federal Advisory Committee Act to encourage and improve events like public hearings.
  • Grants programs within the National Endowment for the Arts, the National Endowment for the Humanities, the Fund for the Improvement of Post-Secondary Education, and the National Science Foundation that promote citizen work.
  • Greater focus on the acquisition of civic skills in the US Department of Education and the Elementary and Secondary Education Act (lately known as NCLB).

Finally, I would describe all the workers who are paid from stimulus funds–including the majority who work in the private sector–as active citizens and would call on them to discuss, plan, collaborate, and serve.

presidents who do too much

A major question in the blogosphere right now is whether the Obama Administration is trying to move on too many fronts at once. For instance, Bill Galston’s New Republic piece entitled “Barack’s Too-Long Wish List” was discussed all over the place and may have even prompted a presidential reply. Obama said, “When we issued the budget, … they said, ‘Boy, these Obama people, they’re really ambitious. They’re taking on health care. They’re taking on energy. They’re taking on education. Don’t they know that there’s this bank crisis right now? We’ve got to do one thing at a time.'” The President then argued for addressing education, energy, and health care along with the financial crisis.

There haven’t been all that many new presidents in American history, so it’s hard to know empirically how they should act in their first six months. Our only evidence is a small set of case studies. Galston mentions Jimmy Carter, who “sent a flood of proposals down Pennsylvania Avenue, so many that Congress soon bogged down in near-gridlock. By the end of his first year, American[s] were beginning to wonder whether Carter could get things done and–worse–whether he was up to the job.”

Franklin Roosevelt is famous for his ambitious agenda, but Galston notes that: (a) things were dramatically worse in 1933 than today, and (b) FDR actually focused first on the economic emergency and then used his early successes to build momentum for other reforms.

I don’t know how many initiatives is too many, but I’d observe that the number of different topics that an administration addresses (call that n) is only one variable that may affect its success. It’s hard to tell whether Carter failed because his n was too high–or whether FDR has his face on the dime because he got n just right. I can think of other explanations for both outcomes.

Specifically, the Zeitgeist was against poor old Jimmy Carter, as we can tell now that the Owl of Minerva has taken flight. Most of the industrialized countries moved substantially right after 1970. Liberals had already enacted the popular parts of the welfare state. They had consolidated prosperity for a majority of their populations, who were decreasingly generous toward the remaining poor. Keynsian policy couldn’t seem to handle stagflation. Liberal coalitions had shattered on the shoals of controversial social issues. Conservatives offered law-and-order and lower taxes, and that was a winning package. The only reason Carter was elected was that Richard Nixon had administered a deadly wound to his own party that took eight years to heal. It was hardly time for an ambitious liberal agenda.

This analysis doesn’t mean that Obama is wise to send a bunch of new proposals down Pennsylvania Avenue. It only shows that there isn’t a very clear historical analogy, and the president has to blaze his own trail. The important questions are: How interconnected are the various topics he’s addressing? How strong and well conceived is each set of proposals? How urgent is each topic? And how much can Congress handle? (But that last issue is only one among many.)

Shih Chieh Huang

Mr. Huang is a youngish artist who uses cheap, discarded objects (soda bottles, baggies) and electronic components such as motion-detectors, LEDs, and fans to create objects that seem animal. My family and I saw his installation at RISD this weekend. It was like a whimsical aquarium; anemone-like creatures on the floor deflated their plastic bags shyly when you walked near them, and the big jelly-fish-like thing in the middle turned to watch you with its human eyes. It all sounds like something that Pixar or Disney would create. I respect their talents, but Huang is a studio artist rather than an entertainer, and I think the difference has to do with the following factors. He has a sense of humor but doesn’t play for laughs. His use of banal waste products to make lovely organic objects stimulates subtle ideas about nature and human action without driving home an obvious point. He leaves the wiring and electronic gadgets unconcealed; there’s no pretense to being something other than an art installation. Most important, Huang is a fine and careful individual craftsman. My little daughter and I were inspired to make something somewhat similar when we got home, and I developed a sense of how remarkably hard it would be to copy a Huang.