how to use empirical evidence

Leave a reply

I’ve written a chapter for a forthcoming book edited by Harry Boyte (Democracy’s Education: A Symposium on Power, Public Work, and the Meaning of Citizenship, Vanderbilt University Press) in which I summarize evidence that colleges and universities can improve the economy by teaching their students civic skills and by being good institutional citizens, participating in local networks for community development.

I think the evidence is reasonably strong. But, like all empirical claims, it has exceptions and caveats. And even if civic education and engagement really do pay economic dividends, something else could work even better: for example, distance-learning, educational video games, or installing surveillance cameras in schools.

Thus we must be careful about how we generate, interpret and use empirical findings. This is where the emerging idea of Civic Studies provides guidance.

Often, social scientists presume that their job is to study a real-world practice that is already fully developed to learn whether—and why—it “works.” Usually they define success in terms of the objectives of the practitioners or their funders. In this case, we would ask whether college-level civic education and engagement generate what politicians demand: jobs.

But nothing simply “works.” Success always requires experimentation, assessment, adjustment, reflection, and new experimentation, in an iterative cycle. By the same token, many things can work if they are developed properly. One could start with civic engagement or with surveillance cameras in schools and improve either one until it enhanced students’ employment prospects.

In the best cases, the researchers who study a given practice are part of the reason that it works. They contribute to its development by offering their data and insights. They choose to work on this practice rather than something else because of a more fundamental commitment. I, for example, have studied deliberation. I want deliberation to work and I hope that the research that I produce will contribute to its success. I have no such commitment to surveillance cameras. I would not study them or strive to improve their impact.

The reason for my hope in deliberation is fundamentally moral. I think a world in which people reason and work together is better than one in which they achieve the same levels of security, income, or welfare without freely collaborating. Deeper down, I believe in a theory that the good life is a life of freedom, reflection, and mutual commitment.

Thus I hope that civic education and civic engagement boost employment because I am fundamentally committed to civic values. My colleagues and I seek evidence of economic benefit to persuade policymakers to support what they should support anyway. If the economic evidence is favorable, we will use it strategically to expand support. If not, our values and commitments should encourage us to improve civic education until it enhances democracy and also produces jobs. Regardless of the empirical results we find, we owe a public explanation of our core values.

A public defense of our values also yields criteria by which to assess the practices that we have been studying empirically. For instance, my chapter for Harry Boyte’s book is about college-level civic engagement (an input) and jobs (an output). I discuss the empirical link between the input and the output, as they exist today. But both are subject to criticism and change.

Today, many civic programs basically take the form of volunteering. But civic education can be reconceived so that it is less about volunteer service than about working on public concerns, where “working” implies serious commitment and accountability for results. “Public work,” in the phrase championed by Boyte and colleagues, means work that is done in public, by diverse citizens, on common issues. Reconfiguring civic education at the college level to look more like public work would satisfy core values that Boyte and colleagues have defended well. It might also strengthen the impact of civic education on jobs and careers. Students would be more likely to learn skills useful for employment if their civic experiences in college were more like paid work.

Meanwhile, jobs could become more public. A given job might serve only the interests of the employer and deny the worker any scope to address community problems in public with diverse other citizens. But even if the employer is a for-profit firm, the job can promote and encourage public work. For example, I presume that the corporate executives, government officials, and labor leaders who attended meetings of the Lehigh University board contributed insights from their daily work to the conversations about Lehigh and Allentown. They then brought ideas from those discussions back to their jobs. If that is true, they were doing “public work” in the Lehigh boardroom and in their own offices. Public work is obviously harder for low-paid service workers and low-ranking bureaucrats, but within many industries and professions, a struggle is underway to recover their public and democratic traditions.

If we made civic education into public work and also created jobs of greater public value, then the alignment between civic education and employment would be stronger and we would find more impressive evidence of economic impact. The data would then satisfy governors and presidents who want to see colleges produce jobs. More importantly, we would be building a better society and the educational system to support it.

qualms about Behavioral Economics

Leave a reply

In Sunday’s New York Times, Katrin Benhold describes how the current UK Government has embraced “behavioral economics.” The Cameron Government has been influenced by Richard H. Thaler’s and Cass R. Sunstein’s book Nudge to adopt policies like telling people who are late with their taxes how many of their neighbors have already paid. A government can improve your behavior (citizen) by showing that it knows what you are doing, by sharing that knowledge with your fellow citizens, and by demonstrating your similarity or divergence from the norm.

Another example is Mayor Bloomberg’s ban on huge sodas. You can still buy as much soda as you want in New York, but limiting the individual portions to 16-ounces confronts you with an explicit choice if you decide you want to drink more than that. Proponents say the government can get better results with less force by using such techniques, and they call it “libertarian paternalism.”

Real libertarians are not happy. Sean Collins writes in the libertarian magazine Spiked:

This paternalistic approach changes the relationship between government and citizens. Instead of government representing us, working for us, government now works on us, trying to change our interests. It would be one thing if government sought to convince the public in open debate, but those who would nudge or ban do not want to have open debate or discussions. As the term ‘paternalism’ implies, citizens are essentially treated like children who do not speak; they are only spoken to.

Followers of Michel Foucault are not libertarians, but they should be equally concerned. They should recall Foucault’s discussion of the “panopticon,” Jeremy Bentham’s scheme for a prison designed so that each prisoner can be observed at all times but cannot tell whether he is being watched:

“So it is not necessary to use force to constrain the convict to good behaviour, the madman to calm, the worker to work, the schoolboy to application, the patient to the observation of the regulations. Bentham was surprised that panoptic institutions could be so light: there were no more bars, no more chains, no more heavy locks ….

The Panopticon was also a laboratory; it could be used as a machine to carry out experiments, to alter behaviour, to train or correct individuals. To experiment with medicines and monitor their effects. To try out different punishments on prisoners, according to their crimes and character, and to seek the most effective ones. To teach different techniques simultaneously to the workers, to decide which is the best. ….

Although it arranges power, although it is intended to make it more economic and more effective, it does so not for power itself, nor for the immediate salvation of a threatened society: its aim is to strengthen the social forces – to increase production, to develop the economy, spread education, raise the level of public morality; to increase and multiply.” (Foucault, Discipline & Punish)

In defense of the “nudge” idea, I would say that governments have always influenced how choices are presented. It makes sense to be deliberate about the design of choices. I am fine with making you decide to buy soda 16-oz at a time.

Yet there are good reasons to be skeptical about behavioral economics as a tool of governance. Your overall reaction will depend on what most deeply concerns you. You may think that our main problem is unhealthy or immoral personal behavior–people failing to pay their taxes, for example, or drinking 32 ounces of sugary soda at a time. You may, furthermore, believe that to change their behavior by banning or taxing it is often too costly in terms of individual freedom, burdens on the state, or sheer cash. Then it will be appealing to use behavioral economics to influence citizens’ choices, just as it was very tempting to build state prisons according to the principles of Bentham’s panopticon. One guard, very few beatings and executions, yet everyone behaves.

A different stance begins with the idea that modernity poses a threat to the human being as an end-in-herself. Modern rationality is means/ends rationality: we constantly develop and refine tools for getting other people to do what we want, whether those tools are laws and surveillance, bureaucratic files, surveys, advertisements of all kinds, payments and rewards, or taxes and penalties. Each of those devices whittles away at people’s capacity to decide for themselves how to live. From that perspective, manipulation is a fundamental problem, worse than obesity or tax-evasion, and behavioral economics is just the latest and most sophisticated version of it.

Benhold uses the verb “manipulate” in her basic description of the British behavioral economic policies:

Manipulating behavior is old hat in the private sector, where advertisers and companies have been nudging consumers for decades. Just think of strategically placed chocolate bars at the checkout counter. But in public policy, nudge proponents study human behavior to try to figure out why people sometimes make choices that they themselves would consider poor. Then they test small changes in how those choices are presented, to see whether people can be steered toward better decisions — like putting apples, not chocolate bars, at eye level in school cafeterias.

It is better to eat apples than chocolate bars. And it is appropriate for a school to shape students’ behavior. But the classical republican ideal is that no one may influence your thoughts and actions without giving you an explicit justification, and you must have the right to respond if you don’t agree. No one can say, “Do this because I say so.” Your response to being coerced may be as modest as voting against the people who are trying to regulate you, but the exchange of reasons (on their part) and actions (on yours) respects your dignity. Moreover, to the greatest extent possible, the citizens of a republican regime must decide how to constrain and improve themselves and create their own norms. Each is accountable to the others, and nobody manipulates us.

In We Are the Ones We Have Been Waiting For (pp. 60-2), I discuss more democratic ways to change our behavior:

Human beings are distinctive because we can have ordinary desires plus desires concerning our desires. For example, I may want to put down a difficult scholarly book that I am reading so that I can watch a trivial television show, and at the same time want not to have that desire. … If I turn myself into someone who enjoys scholarly books more and trivial TV shows less, I am not only entitled to believe that I have done the right thing with my time, but that I have also improved myself. In that way, the self (personal identity) is connected to second-order volitions.

I introduce this concept here because we are capable of assessing and altering our own second-order volitions in ways that produce conscious development, not just random change. In the words of the Port Huron Statement that inaugurated the New Left in America, we “have unrealized potential for self-cultivation, self-direction, self-understanding, and creativity.” The Statement proceeds to note that this process of self-cultivation is not individualistic, on the model of a Romantic artist developing his or her own genius. “This kind of independence does not mean egoistic individualism—the object is not to have one’s way so much as it is to have a way that is one’s own. . . . Human interdependence is contemporary fact.”

Indeed, most of the effective techniques for improving our second-order volitions are “relational” and collaborative. Religious congregations, Study Circles and other deliberative forums, internally democratic associations, and participatory social movements convene people to decide on what they should want and then to hold one another accountable for changing their identities by changing what they want. …

Meanwhile, as a whole country, we have both desires and second-order volitions. We want to drive our big SUVs to work, and we want to be the kind of country that does not want to do that. Whereas individual consumer choices elicit our ordinary desires, civic acts such as making arguments in public and voting make us think about our second-order volitions. A good law is not a reflection of what we want but of what we think we should want.

Again, I do not rule out the possibility that a democratically elected government might put apples on the lower shelves of school cafeterias, ban 32-oz sodas, or even inform tax scofflaws how many of their neighbors have paid on time. But each of these acts is a potential threat to the dignity of the persons being regulated, and so it requires explicit public discussion and careful review.

I realize, by the way, that I have combined allusions to libertarianism, civic republicanism, and Foucault in this post, even though they represent very different perspectives. But their differences emerge mostly in what they say about how we should govern. Presented with “libertarian paternalism,” I think they would converge on the same hostile response.

(See also “the new manipulative politics: behavioral economics, microtargeting, and the choice confronting Organizing for Action” and “qualms about a bond market for philanthropy” for similar concerns about another popular idea, social investing.)

civic engagement: the claymation video

Leave a reply

(Dayton, OH) Edgar S. Cahn has been trying to improve the relationship between citizens and governments since 1964, when, as executive assistant to Sargent Shriver, he helped to implement the mandate to achieve the “maximum feasible participation” of the poor in  anti-poverty programs. Later he founded the Citizens’ Advocate Center and co-founded the Antioch School of Law, helping to invent clinical law programs. And now, to illustrate his new book No More Throw-Away People: The Co-Production Imperative (which I haven’t read), he has produced this great claymation video on engaging citizens. (I believe the metaphor of squares and blobs, referring to formal organizations and loose citizen groups, originated at the Kettering Foundation, which I am visiting today.)

The Parable of the Blobs and Squares from James Mackie on Vimeo.

ALEC’s secret documents and our political economy

Leave a reply

On a flight to Philadelphia, I am reading the secret documents from ALEC (the American Legislative Affairs Council) leaked to the Guardian. Although they contain some juicy details about declining membership and internal controversies, I find some critical interpretations of these documents a bit overblown. For instance, ALEC will require its state chairs–who are also legislators–to pledge: “I will act with care and loyalty and put the interests of the organization first.” This is being treated as evidence that ALEC’s state chairs must put ALEC before their own constituents and consciences.  To me, it just sounds like boilerplate.

But the documents do give a window into money and politics. Consider that:

Businesses have common interests in lower taxes and weaker regulation. But they face a Prisoner’s Dilemma collective-action problem. Generic “business-friendly” legislation benefits them all (a bit), yet fighting for such legislation costs each firm money and reputation.

The solution is one organization that collects dues from many members and acts in their common interest. But …

Members will be tempted to leave if their narrow interests aren’t prominent enough in the collective agenda. The internal documents show that Coventry Health Care dropped ALEC because it had “joined for a single issue” (presumably health insurance) and Novo Nordisk Pharmaceuticals may have left “because only interested in diabetes related issue.”

By Robert Michels “Iron Law of Oligarchy,” any coalition will be taken over by a relatively zealous few with passionate commitments to ideological values. Dana Milbank writes, “When I first dealt with ALEC as a state government reporter 18 years ago, it was right of center but known for thoughtful policy research. But it has since adopted an aggressive agenda to pass legislation expanding gun rights and voter-identification requirements, and limit the reach of public-employee unions, social-welfare programs, consumer and environmental protections, and Obamacare.”  Not all these issues are in the economic interest of corporate backers, and some have left as a result. The ALEC documents specifically note that many financial services companies left the coalition “Due to controversy.”

Further, ideological disputes may divide potential supporters. The Solar Industries Association “left [ALEC] because their bill did not pass the task force.” (I’ll bet their self-interest in federal support for renewable energy met ideological opposition within ALEC.) The Pioneer Institute was “kicked out of ALEC (?) because of education issue.” Pioneer opposes the Common Core standards on principled grounds that I don’t happen to share, although I respect them. But ALEC supports the Common Core, which has business backing. So ALEC expelled Pioneer.

A coalition can operate more effectively in secret, because members can solve some of their collective-action problems through private negotiations. For instance, if a bank agrees to support conservative social legislation in return for deregulatory policies, that will look terrible. But if such a deal can be arranged privately, it may work. Hence ALEC’s tradition of private meetings.

Even so, the collective-action problems are tricky, and the organization is vulnerable to rules imposed from outside. For instance, according to the documents, “ALEC does not wish to be perceived as a lobbying organization and therefore does not wish to register as a lobbyist in any state.” To register as a lobbyist would bring criticism on its corporate members and trigger disclosure requirements. This means that stronger campaign-finance and lobby-disclosure laws might hamper ALEC somewhat.

In any case, a business coalition that spends more than $1 million on its own staff’s salaries must consider the odds of raising money for any given cause. ALEC’s documents indicate that it is considering working on Native American Tribal Issues, but “there may be little to no private funding for this issue.” It is also considering issues that affect travel and tourism, but “individual companies that join will very susceptible to dropping ALEC if there is public pressure.” (Presumably, tourist industries rely on discretionary consumer choices and could alienate customers easily.)

There can even be such a thing as too much money. The documents suggest that ALEC is a little wary of advocating on behalf of casinos, because “according to Opensecrets.org, the gaming industry contributed directly over $64 million in the 2012 elections and $32 million lobbying expenses. … This industry could potentially out fund other industries at ALEC.”

(By the way, I love the irony that ALEC consults opensecrets.org–a good-government watchdog group–to fine-tune its own lobbying agenda, but it does not disclose any of its own activity as either campaigning or lobbying.)

Although I have itemized some challenges confronting industry when it attempts to influence government, the challenges are much worse for, say, homeless people or poor families. So none of the above is meant to suggest that the political system is fair. The ALEC documents still offer interesting insights into our political economy.

assessment and accountabillity for civics

Leave a reply

These are some notes for a presentation I will make later today at the New England Association of Schools & Colleges conference. NEASC is one of the six regional accrediting associations in the US. It works by “developing and applying standards, assessing the educational effectiveness of pre-school, elementary, middle, secondary, and postsecondary educational institutions.”

As measurement and accountability have become more important at all levels of education (from pre-K to graduate school), the measurement of civic outcomes has generally been forgotten. It is not clear that civic education has been dropped as a result. All states still have some kind of civic education requirement at the k-12 level. Most colleges still have programs that emphasize service or activism. However, levels of attention, innovation, and investment have clearly suffered because we do not measure civics very seriously.

Measuring anything valuable and complicated is a challenge, and trying to improve any form of education by imposing measures from the outside is always somewhat problematic. But measuring civic education raises special challenges:

  1. Civic engagement is intrinsically interpersonal. Being a citizen means relating to other citizens and to institutions. Measures of individual civic performance (such as multiple-choice tests, essays, or surveys of individual behavior) may miss the point altogether.
  2. Citizens engage on current issues that are often local. That means that the topics of their engagement vary and change rapidly. Standardized tests of civics–simply because they are standardized–must emphasize abstract and perennial questions (such as the US Constitution) and omit equally important current and local matters.
  3. Civic engagement can be either good or very bad, depending on the means, methods and objectives of the participants. Margaret Mead said, “Never doubt that a small group of thoughtful, committed citizens can change the world; indeed, it’s the only thing that ever has.” But Mussolini and his fellow fascists started as a small group of thoughtful and committed citizens. They changed the world for the worse. Measures of activity or impact that are value-free fail to distinguish between fascists and Freedom Riders.
  4. In many fields, we can decide what students should learn by assessing whether they are prepared to succeed in their chosen profession or in the labor market more generally. For instance, good engineering education makes good engineers, and good engineers are those who succeed in engineering jobs. Likewise, good citizens succeed in democracy and civil society. But what “success” as a citizen means is controversial. That is what radicals, liberals, conservatives, libertarians, patriots, cosmopolitans, Greens, and others argue about: what we owe to each other (and to nature and future generations) and how we should relate to the community and the state.
  5. When assessing education overall, it makes sense to ask whether it enhances the long-term well-being of the students, which can be measured in terms of earnings, health, or psychological flourishing. Some evidence suggests that being an engaged citizen boosts such outcomes. For instance, being able to define and address problems with peers is a civic skill that can also pay off in the labor market. Contributing to your community can make you happier. But the relationship between being an excellent citizen and flourishing as an individual is complex. In his great book Freedom Summer, Doug McAdam shows that the volunteers paid a severe personal price for their efforts to register Black voters in Mississippi in 1963. They were worse off than a comparison group in terms of happiness, career success, and health ten years later. That is no argument against the Freedom Summer program, which wasn’t meant for their benefit. It was one part of a glorious struggle against Jim Crow. To measure it in terms of the developmental benefits for the participants would have been a travesty.

I think it’s essential to measure civic education in an era of assessment and accountability–if only so that educators and students can track their own progress. Assessments must be interactive, not private and individual. Evaluation must consider ethics and values; it is not enough to act or to affect the world–you have to make it better. The question of what to measure is somewhat controversial because it relates to questions about what kind of society we should have. But there is a lot of common ground and room for compromise. In any event, we should decide what makes a good citizen not by asking what skills pay off in the marketplace or what civic activities boost students’ welfare. We must start with a theory of the good democratic society and then ask what skills, values, knowledge, and commitments we need from the next generation of citizens.

In my recent book, We Are the Ones We Have Been Waiting For, I argue that citizenship  fundamentally means: (1) deliberating with other citizens about what should be done, (2) actually working with other people to address problems and reflecting on the results, and (3) forming relationships of loyalty and trust. That theory derives from my study of politics, not primarily from a theory of education or youth development. I argue that the US political system depends on these three aspects of citizenship, all of which are in decline for deep, structural reasons. If I am right, these are the attainments that we should try to teach, and our measures should capture whether people can (1) deliberate, (2) collaborate, and (3) form civic relationships. If I am wrong, the counterargument should be a different theory of what our society needs from its people.