Monthly Archives: October 2012

polls as truth, polls as strategy, and what that tells us about social knowledge

I am a pollster. My organization, CIRCLE, just released a national survey of young adults that shows Obama ahead of Romney among likely voters under 30: 52% to 35%.

Pollsters much more prominent than I are under fire. Look at the comment thread on any high-traffic blog post or news article that reports a poll and you will see fervent remarks denouncing the survey for deliberate, partisan bias. Typically, the charge is that a poll showing Obama ahead has been conducted to help Obama (even though one might think that a lowball estimate would work better, by alarming his supporters into voting).

One sees blanket denunciations as well as very precise, faux-erudite critiques. For instance, the Detroit News showed a pro-Romney bias because it has a libertarian editorial board. Nate Silver is cooking the books because the Times is liberal. National polls are biased to Romney because they miss cell phone users. I was on talk radio yesterday in San Francisco, and a caller argued that Obama’s support cannot have declined because of the first presidential debate. Instead, his decline in national polls must be a deliberate distortion to set up the fraud that will occur on Election Day, when Romney will use doctored voting machines to steal the vote.

I would like to say: Who you want to win is different from who’s ahead. The former is a value-judgment; the latter is an empirical proposition. Empirical propositions are true or false. So just because you don’t like a poll, it doesn’t mean it’s wrong.

This is largely true, but won’t quite suffice.

I do believe that the average of the polls provides an accurate a picture of the horse race. Aggregating all available surveys produces a gigantic sample, and using lots of pollsters’ data reduces the error caused by their specific methods. We said that youth favored Obama by 52%-35%. We had interviewed a randomly recruited online sample from Knowledge Networks. The very same day, completely independently, and using a random-digit-dialing survey of land-lines and cell phones, the Pew Research Center pegged the youth vote at 56%-35%, well within the margin of error of our result. I take this as confirmation of our finding. I do not read it as mere coincidence, because the same thing happens every day. Separate pollsters draw modest random samples of Americans, use different questions and modes of contact, and come up with quite similar results. The method works.

On the other hand:

1. There is no truth now about how people will vote next week. If the polls are supposed to be predictive, that’s not a typical empirical truth.

2. Each poll requires a whole set of choices that affect the findings. We hired Knowledge Networks, which randomly recruits a national sample and provides people with free Internet access if they need it. We drew a random sample of their panel with large minority sub-samples that we adjusted to make the sample resemble the Census demographic profile of 18-29 year-old citizens. We asked respondents: (1) how likely they were to vote, (2) whether they were certain to vote for Obama, and (3) whether they were certain to vote for Romney. (We randomized the order of the two latter questions.) To calculate Obama’s share of the youth vote, we reported the proportion who said that they were extremely likely to vote, they preferred Obama, and they did not prefer Romney.

The alternatives are myriad. If you call people by phone, you are likely to give them a choice of the candidates and code people as undecided only if they refuse to respond. Often, the interviewer pushes back and says, “If you had to choose …?” That reduces the undecided rate, which was fairly high in our poll. The survey can just ask about Obama and Romney, or it can add two minor party candidates (Libertarian Gary Johnson and Green Jill Stein), or it can add even more options. We accepted a generic response about “someone else,” and four percent chose that.

It is typical of social science that truth is somewhat obdurate–not just made up, but stubbornly out there–yet reality is very much colored by our methods and choices.

3. None of us directly calls more than 1,000 Americans to interview them about the presidential election. We trust other people to do that for us. You trust me and my colleagues to survey youth–or I hope you do. We trusted Knowledge Networks to draw a good sample for us. I trusted my colleagues to run the numbers right. Since social knowledge is mediated, it relies on trust of strangers or of institutions, or both. This was the problem for the caller in San Francisco. He took as a premise that Bush had stolen the 2004 election, which would require a very large conspiracy. If that’s afoot, then all the surveys in Nate Silver’s model could be deliberately distorted to show Romney gaining after Oct. 4. Clearly, I do not agree with this, but then again, I would be part of the conspiracy, so why listen to me? More seriously, trust is fundamental, and excessive or automatic trust is foolish. So the questions for all of us are: whom to trust, how much, and when?

a university’s impact on society

An important discussion is underway at Tufts about how to enhance and evaluate the university’s impact on society. Here is my emerging view:

A university can have “impact on society” by doing things that benefit people outside of academia. For example, Tufts veterinary scientists contributed to eradicating rinderpest, also known as cattle plague, which had caused suffering among nomadic peoples around the world. We should be proud of that kind of impact.

But a different kind of impact occurs when faculty, students, and staff engage with people outside the university in productive dialogues or collaborations that benefit both sides. I see three reasons to emphasize that kind of “impact” in our strategic planning, even as we continue to celebrate and support academic work that simply benefits society.

First, participating in and strengthening public dialogue is a moral imperative. A university has no right to define good impacts unilaterally, to choose priorities for any society, or to select appropriate means to attain desired outcomes. These are questions of value, and other people (from Medford, MA to the globe) must help to decide them. But because we handle ideas with sophistication, we can strengthen the public dialogue.

Second, “impact” encompasses the whole university if it is understood as intellectual and practical engagement with society. Liberal arts professors are unlikely to discover the equivalent of the rinderpest vaccine–a concrete benefit that can be transferred from the university to society. But liberal arts professors and students certainly contribute to public dialogue and debate.

Third, if impact is defined as public engagement, it is a comparative strength of Tufts. To be sure, many fields–from bioethics to international development, from museum studies to city planning–are now focusing on public dialogue and collaboration. Therefore, most universities now employ people who are interested in engagement. But Tufts is particularly rich in such work. This highly selective list of examples could easily be expanded:

  • The Tufts Community Research Center has won several large federal grants for sophisticated science. TCRC has a unique governing committee composed equally of Tufts faculty and community leaders; all of its projects involve collaborations with community groups, and many of its research grants originated with ideas from NGOs.
  • Classic professor Gregory Crane, editor-in-chief of Tufts’ Project Perseus and a pioneer in the digital humanities, has shown that classics offers excellent opportunities for diverse people to construct new knowledge together. For example, Crane enlists Muslim seminary students in the Middle East to help annotate the classical Arabic texts that are essential for our understanding of Greek sources.
  • The Lincoln Filene Center for Community Partnerships within Tufts’ Tisch College of Citizenship and Public Service cultivates durable and reciprocal partnerships with NGOs and public agencies in Medford, Somerville, and Boston’s Chinatown. These partners then collaborate with Tufts on teaching and research projects.
  • The Department of Urban and Environmental Policy and Planning, the Department of Public Health and Community Medicine, the Eliot Pearson Department of Child Development, the School of Dental Medicine, and the Jonathan M. Tisch College of Citizenship and Public Service are among the departments, programs, and schools at Tufts whose formal mission statements highlight public engagement.

If this analysis is correct, it raises some interesting questions about how to encourage, strengthen, and assess “impact” across a whole university. I don’t have the answers, but I would want to head off several misconceptions.

First, impact is not simply about expanding the scale of our engagement. We do not necessarily want more of our faculty to become “talking heads,” influencing public discussions through the mass media. That can be valuable, but it is not appropriate for everyone. Engagement is important at all scales and in many venues.

Second, engagement is not “service,” understood as an additional requirement above research and teaching. Engagement is a dimension of research and teaching that can strengthen a scholar’s core work.

Third, engagement does not mean the same thing across a university that promotes the liberal arts, the applied sciences, and several professional disciplines. But engagement can have significant (if diverse) meanings for all those parts of the university–and can help bring them together.


Obama at 52% and Romney at 35% among youth

CIRCLE today released a poll of young people’s views of the election. Our survey, commissioned by the Youth Education Fund, is unique in that it polled 1,695 youth (ages 18-29) in June/July and 1,109 of the same youth between October 21 and 23. Surveying the same people twice provides powerful evidence of change over time.

  • The proportion saying they are extremely likely to vote has risen 9.9 points, from 44.7% to 54.6%. Two-thirds (67.3%) of young adults are “very” or “extremely” likely to vote, up 7.1 percentage points since June/July.
  • The proportion who are paying attention to the election has also risen, from 56.1% to 71%.
  • If the election were held today, Obama would win the youth vote by 52.1% to 35.1% among those registered voters who are “extremely likely to vote.”

My quote from our press release: “The conventional wisdom holds that youth enthusiasm is down compared to 2008,” said CIRCLE Director Peter Levine. “But intent to vote is rising fast. President Obama has a majority of likely young voters behind him, but a significant proportion are open to voting for Governor Romney, who has a clear opportunity to improve over John McCain’s record-low support in 2008.”

I will be on MSNBC around 11:30 am eastern today, on WGBH-Boston’s Innovation Hub at around 1 pm today, and on KALW-San Franciso at 10 am Pacific tomorrow to discuss, and later I will post the audio or video here.

[Update: actually, MSNBC cancelled to cover Sandy 24/7. But the WGBH discussion with host Kara Miller and the Pew Research Center’s Paul Taylor is available online, here. And the KALW discussion with Ali Budner (the host) and Lee Rowland of the Brennan Center and Tova Andrea Wang of Demos is here.]

Carolyn J. Lukensmeyer, Bringing Citizens Voices to the Table

Carolyn Lukensmeyer has published Bringing Citizen Voices to the Table. Carolyn is Executive Director of the National Institute for Civil Discourse and founder and former President of AmericaSpeaks. She also has extensive experience in state and federal government. Her book distills the lessons of her career into seven strategies that can be adopted by public managers at all levels of government. If they implement these strategies, it will strengthen our democracy.

I know of no other book that provides the same recipe, and it is grounded in general principles and evidence plus compelling case studies and examples. Carolyn also argues for large-scale policy changes that would enable public managers and the public to use the strategies that she recommends. Most of the book is analytical and dispassionate, but she incorporates autobiographical material that will make it more approachable for practical readers. In places, it is quite moving.

The most immediate audience is public managers, from city managers and planners and school district superintendents to state and federal agency officials. Another large audience consists of people who may be considering becoming public administrators, especially graduate students of public administration. Organizers of deliberative democracy should also read it, and it deserves  an international audience, because the same strategies have also been tested in other countries–and in some cases, are better supported overseas.

See a nice summary in the form of a recent Nonprofit Quarterly article entitled “The Case for Citizen Engagement.

Call 800-356-5016 and use Promo Code CL252 to save 36% on the cover price of the book.

campus speech codes and college student voting: or, how to make an empirical claim in the Times with no evidence

Greg Lukianoff, president of the Foundation for Individual Rights in Education, is an opponent of campus speech codes. In today’s New York Times, he argues that these codes “discourage civic engagement” among college students. Lukianoff is entitled to his opinion of “stringent speech codes.” But his op-ed makes an empirical claim–that speech codes reduce students’ voter turnout–with a breathtaking lack of evidence that makes me wonder why the Times accepted his piece.

Lukianoff opens with this claim: “Despite high youth voter turnout in 2008 — 48.5 percent of 18- to 24-year-olds cast ballots that year — levels are expected to return to usual lows this year, and with that the usual hand-wringing about disengagement and apathy among young voters.” The relevant number is actually the turnout rate of currently enrolled college students, which was 59.7% in 2008. As our graph shows, that rate has been quite stable for 40 years, so it is unlikely to “return to usual lows” in 2012.

Further, college students voted at a much higher rate than their non-student contemporaries. That doesn’t disprove Lukianoff’s claim that colleges “do as much to repress free speech as any other institution in young people’s lives.” It could be that colleges repress free speech and voting, but other factors associated with attending college, such as higher socioeconomic status, boost turnout more. To isolate the effects of the speech codes, we should compare colleges that have stringent speech codes to those without. Lukianoff hasn’t done that, and neither have I. We do, however, know that Tufts students voted at around 90% rates in both 2004 and 2008, even though Tufts has a “red” (bad) rating from the Foundation for Individual Rights in Education. Meanwhile, the University of Pennsylvania–a rough peer of Tufts–gets a green rating and has a very similar voter turnout rate as ours: 89.6% in 2008.

CIRCLE is working on a tool that will allow colleges to measure their turnout accurately if they choose to do so. That will make this kind of research much easier and more robust. Meanwhile, the limited available evidence suggests no correlation at all between the presence of a campus speech code and turnout. I wouldn’t try to publish that null claim, because the evidence is too thin, but I don’t see how the Times can publish an article that rests on the contrary assumption when there is no evidence whatsoever for it.

By the way, I have mixed feelings about the speech codes themselves. Free inquiry is a profound value. On the other hand, a central mission of the university is to select and promote high-quality speech. We constantly evaluate and filter speech when we decide whom to hire, admit, and invite to campus, how to grade students’ work, which courses to approve, which groups and events to fund, and which books to buy for the library or print through the university press. In making those judgments, we ought to be guided by J.S. Mill and Alexis de Tocqueville and make sure that we are not suppressing unpopular views that would contribute to the dialogue. At the same time, we are not governments that must recognize almost untrammeled freedom of speech as a human right. Within our communities, we are entitled to balance individual freedom against criteria of quality, properly defined. More on that here.