what must we believe? « Peter Levine

A study by Roland G. Fryer, Jr. provoked controversy because of its conclusion that Black and Hispanic people are not more likely than whites to be shot to death by police, although they are more likely to experience non-lethal force at the hands of police. In “Race, Policing, and The Limits of Social Science,” Lily Hu uses this study and the resulting controversy to explore important questions about the persuasiveness of social science. Her questions are not merely academic–they have existential significance as we each decide how to interpret and improve the world.

Speaking for myself (but in a similar vein to Hu), I would draw attention to the results in the tables near the end of Fryer’s piece (pp. 39ff). You’ll see familiar-looking columns of numbers, many of which are means or coefficients. They are labeled with text, including such words or phrases as “High-crime Area,” “Carrying Suspicious Obj,” “Incident type: Street Stop,” “Officer Unit Majority White,” “Pinellas County,” and “Hispanic.”

Why are we reading these tables? For one thing, Roland Fryer has captured our attention and a baseline level of trust. This is not a given. I crunched some numbers on police discrimination last year and drew modest traffic. I am not complaining: Fryer genuinely deserves much more attention because of the ambitiousness and originality of his paper (even if one criticizes it). Yet it is worth noting that we read his paper as a consequence of many causal factors, including Fryer’s talent and hard work but also his status and position.

In short, knowledge results–like everything else–from causes. The factors that cause us to know and to trust any given claim include power and social status.

Fryer’s specific findings are the result of methods that social scientists have developed and that he chose to use. Methods are always contestable, and, in this case, they have been challenged. I do not have anything valuable to add about his methods, but their contestability is important.

As for the data that Fryer uses, they result from social processes. He analyzed five million records from New York City’s Stop, Question, and Frisk program, which mandated a great deal of data-collection as part of its approach to enforcement/social control. He also used the Police-Public Contact Survey, which is a federally funded survey conducted by the US Census Bureau for the Justice Dept. Finally, he and a team coded reports from the files of the police in LA County, Houston, Austin, Dallas, Houston and six large Florida counties.

Thousands of people generate these data: randomly selected residents who complete surveys and police officers who file required reports. Many hundreds of people design these instruments and make sure that they are completed. A police officer files a report about an incident with a civilian because of other required reports: a personnel file on the officer, an evaluation of the unit commander, etc. Deeper in the background are previous efforts to measure our social world. For instance, racial categories come from thousands of previous surveys designed and fielded for other purposes. We code and analyze stored bodies of text (such as police files) thanks to techniques previously developed for other research.

Different phenomena could be measured, and different measures could be used. The government of France does not record race and literally does not know how many French residents are people of color. In contrast, the US has always categorized our population by race, albeit with changing categories and purposes. Already in the Census of 1800, Americans were categorized as free whites, other free persons, or slaves.

One way to think about such differences is in terms of choices. We could choose not to measure race, to measure it in a different way, or to measure things that we do not measure now. But this “we” is misleading. We citizens are not convened like a committee to review discrete choices. Instead, the practice of measuring any given thing often results from concerted efforts by specific people or groups. Some people develop and revise concepts and organize and advocate; other people then spend money or apply power to cause data to be collected. I have been involved in such efforts–for instance, helping to write the federal assessment of civics, and playing a role in generating voter turnout data for most US colleges and universities. These are purposive efforts, undertaken to change the world for specific reasons. They reflect people’s values and strategies.

People also undertake concerted efforts to build up social institutions. It is not natural or inevitable that we have police at all–meaning uniformed, armed, bureaucratically organized public-sector employees with unique rights and responsibilities. Having police is a choice, but again, it is not an item on a menu that was set before the public. Instead, policing is an institution with its own inertia, constituencies, and political influence. The choice that each of us faces is whether to support policing, assent to it, subvert it, or help to build up an alternative.

The same is true of the political jurisdictions listed in Roland Fryer’s tables. It is not inevitable that Houston and Pinellas County, FL are organized as entities with police forces and other state powers. Indeed, Elinor Ostrom showed that much smaller jurisdictions produced better policing, and she found especially good results in small Black-majority communities that had their own police forces. So it is a choice to have a Houston police force–but again, this choice reflects many decades of concerted efforts by many people with many agendas.

Particular facts are the results of power deployed through social institutions. Power is not necessarily bad: it is nearly synonymous with “capacity,” and we want capacity. Nor is power necessarily zero-sum: we can measure and value more than one thing at a time. Therefore, to say that knowledge emerges from power is not cynical. Whether knowledge is good or bad depends on the motives, the means, and the consequences that relate to it.

This implies that to assess knowledge is always partly a matter of values–it is a normative as well as an empirical matter. You can’t assess the crime rate unless you know which activities are deemed illegal and how the law is enforced. You can’t assess the employment rate until you know what counts as employment, which jobs are consistent with a good life, and what happens to the people who are labeled unemployed. The crime rate or the employment rate is not a simple fact: it is a result of social institutions that cause people to behave in various ways and that cause their behaviors to be measured and classified in various ways.

The next question (for me) is: What should we evaluate? I resist evaluating very broad and vague phenomena, such as capitalism or modernity or even the USA. That is a path to ideology, in the bad sense of that word: broad assessments prevent careful thought and nuance.

On the other hand, each social phenomenon is linked tightly with many others. A claim cannot be assessed all by itself. For instance, to accept that a city block is a “high crime area” (as in one of the datasets in Fryer’s paper) is to accept a whole system of social monitoring and control that gives some places that label.

We assess one thing at a time, yet each assessment is related to many previous ones. In this way, we gradually build up a worldview that combines normative judgments, empirical generalizations, causal inferences, and many other components. Hu cites “what philosopher W. V. O. Quine so charmingly called our ‘web of belief.'”

As she points out, a particular piece of information may come along that conflicts with some existing components of our web. For instance, Fryer’s specific statistics about police-involved shootings contradict what I had believed. In such cases, we must consider “what must be sacrificed so that other beliefs might be saved. And since our webs are not all identical, what rational belief revision demands of us will also vary. One man’s happily drawn conclusion (p, therefore q!) is another’s proof by contradiction (surely not q, therefore not p!). Or as the saying goes, one man’s modus ponens is another man’s modus tollens.“

Her rather startling conclusion: “Rejecting a study’s methods or its starting assumptions on the basis of disagreement with its results is a completely legitimate inferential move.”

Confronted by Fryer’s original paper, someone who is very concerned about police shootings of unarmed Black civilians might:

Doubt the sources of the statistics (although the paper uses several different kinds of sources);
Doubt the mathematics, either because of specific methodological concerns or because of a low threshold of trust in the author (who could, in theory, have made basic errors);
Doubt the conclusions because they conflict with other sources of information;
Acknowledge the specific conclusions but accentuate the part of the study that reinforces prior views: police use non-lethal force in a racially discriminatory way;
Modify strategies for police reform to focus more on the non-lethal uses of force; or
Revise basic beliefs, including beliefs about other sources of knowledge, such as news coverage of police homicides.

Hu implies that any of these responses might be rational, depending on one’s overall web of belief. For instance, it would be irrational for me to distrust Fryer’s basic mathematics, because I have accumulated trust in institutions like Harvard and the NBER. They have served me well on many prior occasions. However, other people might have no such basis for trust and might have very well-grounded reasons to doubt a result that contradicts their vivid accumulated experiences. Hu writes:

For those whose beliefs, empirical and ethical, are forged in participation in radical sociopolitical movements from below, to be ill-inclined to accept certain findings about race and policing is to remain steadfast in a commitment to a certain thick set of empirical and ethical propositions in their webs of beliefs: that systems of policing and prisons are instruments of racial terror and that any theory of causation, theory of race, and statistical methods worth their salt will see race to be a significant causal factor affecting disparate policing and prison outcomes. This just is the first test of “fitting the data.” It is not a flight from rationality but an exercise of it.

Bertrand Russell summarized a significant tradition when he wrote his Liberal Decalogue (ten commandments) for educators. He included these points:

1. Do not feel absolutely certain of anything.
5. Have no respect for the authority of others, for there are always contrary authorities to be found.
6. Do not use power to suppress opinions you think pernicious, for if you do the opinions will suppress you.

7. Do not fear to be eccentric in opinion, for every opinion now accepted was once eccentric.
8. Find more pleasure in intelligent dissent that in passive agreement, for, if you value intelligence as you should, the former implies a deeper agreement than the latter.
9. Be scrupulously truthful, even if the truth is inconvenient, for it is more inconvenient when you try to conceal it.

This list is “liberal” in the sense that it prizes autonomy, freedom of conscience and speech for all, a marketplace of ideas, and the individual’s exercise of reason. It is a version of Kant’s “enlightenment.” It has merit–perhaps slightly underplayed in Hu’s excellent article. We suffer from motivated reasoning, intellectual conformity, and polarization, and it takes work to keep our minds open. Russell’s advice is valuable for that reason. In this spirit, we might do well to wrestle with the specific claims in an article like Fryer’s–to see what insights we can get from them.

But Russell’s list is incomplete, for the reasons that Hu presents. Basically, he treats each “truth” and each observer as independent, when they actually belong to Quinian webs of belief and social institutions. Therefore, we should also remember to:

Recognize the consequences or what we choose to say, including arguments and findings that we choose to repeat from other people’s work. In making these choices, strive to do no harm.
When encountering new and troubling findings, don’t forget what we already know or neglect our debt to our existing sources.
When encountering a new argument or study, be open to all of it, not just the headlines. (For instance, don’t forget Fryer’s finding that police discriminate in using non-lethal force.)
Cultivate our whole webs of belief, which ought to be internally diverse and complex but also reasonably coherent.
Value a range of sources of knowledge, including personal experience and testimony as well as statistics and models.
Ask whether it is beneficial–or not–for each of us to speak publicly on any given topic.
Critically assess the institutions that generate knowledge.
Critically assess our own roles in such institutions.
Never neglect the normative aspect of knowledge.
Don’t take the questions for granted, but ask what we could be asking about.