Today’s World Bank’s meeting on community service programs turned out to be mainly a debate about the value of randomized, controlled experiments in evaluation. The Bank wants such evaluations (as does the US Government); many activists and proponents of youth service don’t want to do them. I am also thinking right now about randomized experiments because I will soon play a role in running one in Florida schools.

Experiments don’t work for all purposes; they are not always practical; and they’re not the only legitimate methods.

I did, however, share a positive example of how experiments can be very helpful–in getting out the vote. There were no experiments with voter turnout in the United States for 50 years. That’s surprising because voting is a very simple, measurable act that’s well suited to experimentation. In the mid-1990s, some academics and foundations started pressing the nonpartisan voter turnout groups to use randomized evaluations. The pressure at first seemed unreasonable and even arrogant. It seemed as if the experts wanted randomization for the sake of it.

But now the nonpartisan voting groups are avid experimenters. They are always looking to randomize treatments and investigate the differences in results. When they send out mailings, they design two or more messages and randomize their lists. When they work in a limited number of sites or communities, they choose the sites randomly and reserve others as a control group. They do this even when they are not pressured by funders.

I think there are three major reasons for this “culture of experimentation”:

1. Randomized experimentation is a simple, transparent process. It does not involve elaborate mathematics, which you do need for statistical models. Thus the grassroots groups control their own evaluations. They don’t have to trust outside experts. Experimentation is actually non-technocratic.

2. Random experiments yield useful and counter-intuitive results. For example, going door-to-door is cost-effective even though it costs quite a lot of money per contact. Emails are not cost-effective even though they are cheap. This is good to know.

3. Experimental results can persuade powerful people who are predisposed to be skeptical. The American political parties traditionally assumed that it was a waste of money to mobilize young people. In the 1990s, political consultants often deliberately stripped young people from contact lists to save resources. But the experimental evidence showed that young people would vote if contacted. That led to much more partisan investment in youth turnout. The Obama campaign even has a youth director who comes straight out of the youth voting community in which the experiments were conducted from 1998-2006. Regardless of the experimental data, Obama probably would have campaigned to youth, because he has an appeal with the new generation. Still, it doesn’t seem a complete coincidence that (a) we learn how to mobilize young voters by experimenting, and (b) a candidate captures the Democratic nomination by mobilizing youth.

I’m speaking tomorrow at the World Bank, which has some interest nowadays in the link between civic engagement and youth development. Basically, the idea is that investing in opportunities such as community service will benefit young people around the world, helping them to stay in school, avoid crime and disease, and generally flourish.

We know, indeed, that there are strong positive correlations between service and flourishing for Americans of all ages. Young people who volunteer do better in school; old people are less depressed. To some extent, success may cause volunteering. If you have more money and time, if you’re associated with better institutions, and if you have more confidence and initiative, you are more likely to conduct activities that we call “volunteering” or “service.” For instance, a full-time, white-collar worker may “volunteer” as part of an annual office outing, whereas a homeless person is unlikely to do something that would be called “volunteering” (although he might help his peers).

To an important extent, the reverse is true as well: volunteering contributes to flourishing. We know this from fairly careful studies, including studies of mandatory service (which avoid the problem that volunteers may be self-selected).

But this is the issue I’d like to raise at the World Bank: “volunteering” does not exhaust civic engagement. Citizens who want to change the world for the better may reasonably select other strategies, including voting, protest, building organizations, convening meetings, or even the age-old tactic of the oppressed, non-compliance. I often cite an example from a CIRCLE Working Paper by Michelle Charles. African American youth in Philadelphia are supposed to be cleaning up graffiti under the direction of middle-class adults. They are dragging their feet, presumably because they think this particular activity is pointless. To me, their service is not civic engagement, but their foot-dragging is. It may have the affect of ending the program, which will be positive social change.

These other forms of civic engagement are not correlated with prosperity and health. Protest, for example, is much more common among Latino and immigrant American youth than among White Anglo youth. That doesn’t mean that protest is bad for individual human development. I don’t think we know enough about its developmental effects (notwithstanding some important studies of 1960s activists). But it certainly could be the case that “volunteering” is good for social adjustment, and protest is bad for individual success, but good for society.

In other words, the strategy of encouraging youth to volunteer has a political agenda to it. Not being radical myself, I may be willing to endorse this strategy; but it’s worth arguing about. Certainly, many of the globe’s young people may resist the idea that they should be better accommodated to existing institutions through service.

The fourth National Conference on Dialogue & Deliberation, entitled “Creating Cultures of Collaboration,” is scheduled for October 3-5, 2008 in Austin, Texas. These are huge gatherings of mostly practical people who organize public meetings, dialogs, and discussions. They are also opportunities to discuss deliberative democracy and civic engagement. Early registration is available through May 16th. There are openings for new sessions. Details here.

I’ve spent the last two days helping to advise the government on the design of background questions that will accompany the National Assessment of Education Progress (NAEP), a test-like survey of American kids. The background questions are about demographics and students’ experiences in and out of school. They help researchers to interpret and explain trends and gaps in the academic performance of our students.

The process for designing and administering the assessment is complex. It exemplifies the moral and practical complexity of government, and makes me wonder whether governance could be simpler. (I’m not sure that it could be.)

Take the example of how to categorize people by race and ethnicity. In the 1990s, people who considered themselves “multiracial” started lobbying for a separate category on the Census and other official data-collections. There were some objections, for instance, by people who didn’t want to reduce the count of African Americans because that would affect policies regarding legislative districts. Believe it or not, the Office of Management and the Budget (OMB) contracted with my colleagues and friends at the Institute for Philosophy & Public Policy to decide what to do. OMB recognized that the problem was basically moral, not empirical. My colleagues recommended that respondents be allowed to check as many categories as they liked. There would be no “multiracial” category, but researchers could construct it by counting all the respondents who checked more than one box.

That became federal policy, which means that NAEP must now use a racial/ethnic identity survey that allows respondents to “check all that apply. But that leaves many questions:

♦ Who should answer the questionnaire: students, parents, or educators?

♦ When should it be answered (e.g., while taking a test, or at some other time)?

♦ Must students who have already been categorized now be recategorized according to the new scheme, or can they be “grandfathered in”?

♦ How many separate racial and ethnic categories should be offered? What should they be called (e.g., Hispanic or Latino)?

♦ How quickly must the new scheme be implemented?

♦ How should trends in educational performance be calculated and reported after any changes in methodology? For instance, it has been proposed to separate Asians from Pacific Islanders. Historically, Asians perform much better on the NAEP (on average). Separating the groups will therefore boost Asians’ scores and create a bigger gap, especially between Asians and African Americans and Latinos. The impact of the change, however, will be small. Should the trends in performance be reported without comment, with a huge caveat to discourage the public from paying attention, or not reported at all?

Continue reading →

The memorial celebration for Dad was wonderfully life affirming. All the speakers gave thoughtful, carefully crafted speeches and collectively recalled most aspects of his life, from his skillful and dedicated parenting to his civic participation to his sense of humor. Below are two of his grandchildren and the venue, Syracuse University’s stately Hendricks Chapel. It was a fitting scene in which to recall a man who was one of the world’s experts on neoclassicism. Since he was highly appreciative of Christian cultures and a great fan of churches, the Methodist origins of the chapel did not concern me in the slightest. Still, I was glad to quote the Hebrew words that must have been spoken over his ancestors for a thousand years or more: Olav HaShalom, may he rest in peace, and may his name be a blessing for all who knew him.

Click to listen to the service.