The Coolidge Room in Tufts’ Ballou Hall is a grand Victorian space (1853-4) characterized by unusual free-standing columns of cast iron, thick patterned carpets, potted plants, and oil portraits of the Tufts presidents on the walls. Like any constructed place, it conveys cultural significance–and its meanings are perhaps complex and various–but overall, those hushed old white men on the walls do seem to dominate.

On Saturday, that space was filled with the voice of Professor Sonia Sanchez. I don’t know if her remarks were recorded, but this event at Georgetown gives a flavor of them. She argues, she cites information, she quotes, she analyzes, she remembers, she calls and awaits responses, she echoes and rhymes, she ululates, she sings, she exhorts, she weeps, she laughs.

I was in DC very briefly today to testify on measuring civic engagement in federal surveys. The 2009 Kennedy Serve America Act charged the Corporation for National and Community Service and the National Conference on Citizenship with assessing America’s “civic health.” Congress authorized them to choose the indicators of civic health and to “sponsor a panel of experts, such as one convened by the National Academy of Sciences” to advise. I testified before that panel at the National Academy.

Realistically, we’d be lucky if we could put half a dozen new items on several large-scale federal surveys and keep them going for many years. These items would have to be relatively non-controversial, or else they wouldn’t be politically sustainable. At one point, a federal survey asked, “How many friends do you have, if any?” This did not go over well. Other topics that government agencies probably shouldn’t investigate with surveys include trust in the government itself and the racial diversity of social networks.

Today, Robert Putnam argued that the measures should be neutral with respect to different conceptual schemes and policy debates. He argued, for example, that his own concept of “social capital” should not drive the choice of questions, nor should a survey be designed to assess any current policy, such as AmeriCorps. Good questions would allow multiple theories and policies to be tested over time. That was thoughtful and generous advice, but one needs some kind of conceptual basis to choose items. Otherwise, why not ask about preference for vanilla ice cream?

I had talked about the correlation often found between trust and civic engagement. A federal official remarked that he’d been told that trust was part of civic engagement. He wanted more conceptual clarity from the experts on the topic. His first impulse was to expect the clarity to come from psychometric research. In other words, we would ask numerous questions of the same population and determine empirically which ones cluster together. Those clusters would become the elements of civic engagement; items that didn’t correlate might be dropped.

Psychometrics has value–and has been done in our field–but it won’t settle the question of what to ask. Communities probably need citizens to engage in various ways, including group membership, charity, protest, debate, running for office, etc. Achieving the right amount and proportion of those activities is presumably important. But they need not correlate in individuals. The people who protest need not be the ones who give money to charity.

In short, civic engagement is not one or more psychological constructs. It is a set of behaviors, values, and attitudes possessed by members of a community. One needs a conceptual and normative scheme to decide which ones should count.

To the extent that empirical data are relevant, I think the most important question is which set of behaviors correlates with good social outcomes at a community level. For example, what levels of protest, membership, public work, debate, and charity are socially optimal? But note that the definition of a “good” social outcome is contentious, and we are engaging civically when we debate it. Social benefits are not simply outcomes of civic engagement. Instead, civic engagement affects which outcomes we pursue (and measure). This circularity makes it impossible to pick the measures of civic engagement that are the best predictors of good outcomes for individuals or communities.

My own framework would emphasize activities that involve collectively choosing and then advancing social goals. Deliberative and communicative activities would loom large in my ideal poll. Membership in groups and working on public problems would also be important. I think Bob Putnam was right that no one’s theory should determine the choice of survey questions–and if his won’t, mine certainly shouldn’t. But I suspect that the best we can do is to pool items that reflect as many serious current theories as possible so that we have opportunities to test them empirically. We cannot anticipate theories that may arise later, but we must build the most durable apparatus we can.

[Addition: I should have noted that thanks in part to to the Corporation and the National Conference on Citizenship, the federal government has been measuring civic engagement on the annual Current Population Supplement, so the discussion was not about how to start but how to refine/expand the effort.]