New Jersey, Virginia, and the Law of Small Numbers

It will be tempting to make predictions based on the elections that occur this week: the gubernatorial campaigns in New Jersey and Virginia, the gay marriage referendum in Maine, and the mayoral elections in New York, Pittsburgh, Boston, and elsewhere. People will try to generalize about whether liberals or conservatives have national momentum. They will also want to know whether particular demographic groups are energized or not. For instance, David Kocieniewski writes in the New York Times: “The outcome could also help answer a question nagging at Democrats nationally: Can they keep engaging those drawn to the polls by Mr. Obama‚Äôs charisma and historic campaign, or will last year prove a one-time surge for the party?” Even more forcefully, the Times’ Adam Nagourney writes, “it seems difficult to argue that there are no lessons to be drawn from what happens Tuesday.”

Well, here goes. It is a basic mistake of statistics, a fallacy, to use a few elections to make any generalizations or predictions at all. Each campaign is idiosyncratic; the candidates, strategies, local demographics, local issues, etc. all vary. In even-numbered years, 435 House races occur simultaneously, and the Law of Large Numbers means that the idiosyncrasies should average out and you can say something about the election as a whole. For instance, youth turnout may either rise or fall; the country may move rightward or leftward. But with two gubernatorial elections, there can be no meaningful pattern. To detect a pattern from two cases is to commit the fallacy sometimes called (tongue in cheek) “the Law of Small Numbers,” a.k.a., the “fallacy of hasty generalization.”

To make matters worse, the Virginia and New Jersey gubernatorial elections were not randomly selected. And, as governor’s races, they make especially poor predictors of House campaigns.

Does this mean that you can’t learn anything from case studies? On the contrary, they can be rich sources of information. But you can’t learn from the simple outcome, such as the number of votes cast. You have to look at the details in context and develop insights into what they mean. For instance, Craig Berger criticizes the Corzine campaign for taking youth for granted and expects low youth turnout as a result. His criticism sounds plausible to me and would suggest a general lesson: don’t ignore youth issues. That would be right even if Corzine happens to eke out a victory with youth support.

CIRCLE will release youth turnout numbers the day after the election; here is our press advisory. I think analyzing the turnout is the right thing for us to do, because it keeps young voters in the news and allows people to work with data rather than mere impressions. But I will be trying to argue against generalizing from the two data points we provide.