(Washington, DC for a presentation) The ease with which we can analyze social statistics today is remarkable. There are huge, free data-sets all over the Internet. You can employ exploratory statistical techniques (such as probit models or factor analysis) to fish for relationships. Even easier is to find a column of numbers on a web page, copy and paste them into an Excel document along with a different column of numbers from a different source, and find out whether they correlate. It can take five minutes to accomplish a task that would have taken thousands of person-hours when my Mom was a young health statistician. Many statistical investigations that would have been completely impossible–such as a multivariate model of a Census data-set with tens of thousands of cases–are now quite simple.
Graduate students, faculty, think-tankers, bureaucrats, and even some bloggers are busy at this work every day. But I don’t think we understand society better than we did in 1960. At least, we don’t understand it in ways that help us to make the world better. We are richer as a nation, so we should expect life to have improved. In some respects, it has. Relative to our assets, however, I think our performance is considerably worse. How could rates of high school failure, violent crime, cancer, unemployment, depression, suicide, and poverty be stubbornly flat if we had developed better solutions through social science?
To be sure, bad policy doesn’t imply bad research; maybe there is a problem with implementation or with the motivations of the ruling classes. But figuring out how to address those obstacles should itself be a task for research. Besides, I am not overly impressed by the research-based proposals that are sitting around waiting for politicians and citizens to implement. At best, these ideas seem promising incremental steps, not game-changers.
What’s wrong? It could be that …
1) Correlational research provides limited understanding, because there are always unmeasured factors and influences. More powerful research is always experimental, and we don’t do enough of that. By the way, “experimental research” is not just a matter of randomly selecting treatment and control groups. It also requires bold and exciting new projects or institutions that can be evaluated in that way.
2) We don’t measure the important things. Test scores, yes; students’ wisdom and virtue, no. Voter turnout, yes; emerging political networks, not so much.
3) Our imaginations are too limited by our tendency to take actual facts (“data”) as necessary. Roberto Unger wants economics to be the scientific study of what might be possible. Instead, economics describes the present and recent past and infers from that description immutable laws. These laws may actually be subject to amendment, if we choose to change them.
4) Our attention is focused on the wrong levels or scales of analysis. Perhaps our scale is too modest. We ask whether interventions or programs affect outcomes, not which social philosophy is best. Or our perspective may be too broad and distant. We have tools for assessing whole populations, but few new techniques for understanding–let alone improving–specific neighborhoods, schools, or firms, let alone human beings. (There actually are techniques for those purposes, such as ethnography, asset-mapping, and appreciative inquiry, but they are vastly less influential than social statistics–and I am not sure they are satisfactory.)