measuring what matters

(Washington, DC) I am here for a meeting of a federal committee–one of dozens–that helps to decide which statistics to gather from public school students. We are especially focused on socio-economic “background variables” that may influence kids’ success in schools. What to measure often boils down to what correlates empirically with test scores or graduation rates. For instance, a combination of parents’ income, education, and occupation can explain about 15%-20% of the variance in test scores. And so we measure these variables.

But the mere fact of a correlation between A and B doesn’t mean we should measure both. We could look for correlations between the length of students’ noses and the weight of their earlobes. Instead, we look for covariance between parental income and the total number of questions a kid can answer correctly on a test that we write and make him take. Why? Because of moral commitments: beliefs about what inputs, outputs, and causal relationships matter ethically in education.

So it’s worth getting back to fundamental principles. These would be mine:


First, the quality of schooling (education that the state provides) should be equal, or should actually be better for less advantaged kids. Quality does not mean effectiveness at raising test scores–it means what is actually good. That may include intrinsically valuable experiences, such as making and appreciating art. But quality probably includes effective practices that raise scores on meaningful, well-designed tests.

Second, it’s good when outcomes are equal, but equality trades off against other values, such as freedom for children and parents, and cultural diversity. Also, a narrow focus on equality of outcomes almost inevitably leads to narrow definitions of success and can put excessive pressure on teachers and kids.

Third, individuals’ aptitude probably varies (and the degree to which it varies is an empirical question), but every kid who is not performing very well could probably perform better if he or she got a better education. Thus differences in aptitude do not excuse failure to educate.

Fourth, out-of-school resources affect educational outcomes. These resources vary, and that is not fair. We should do something to equalize kids’ chances. But resources fall into various categories that raise different moral questions:

1. Fungible resources, such as parents’ income or wealth. We can compensate for these inequalities by, for instance, spending more on schools in poor communities. (We tend to do the opposite, but I am writing about principles, not reality.) Note, however, that family income alone explains a small amount of variance in test scores.

2. Attributes of parents that cannot be exchanged or bought, such as their knowledge, skills, abilities, social networks, and cultural capital (ability to function well in privileged settings such as universities and white-collar businesses). It is interesting, for example, that the number of books in a student’s home is a consistent predictor of educational success. This is related to income, but it’s not the same thing. You may be more educationally advantaged if your parents are poor graduate students with lots of books than rich but vapid aristocrats, especially if your parents devote time to you. The challenge is that parental attributes cannot be changed without badly restricting freedom.

3. Prevalent attitudes, such as racial prejudice/white privilege, that may affect students’ self-image; or values relevant to education, such as the belief in Amish communities that a basic education is sufficient. These attitudes vary in how morally acceptable they are. But they have in common the fact that the state cannot change them without becoming highly coercive.

In the end, I think we measure parental resources and their relationship to test scores because we think that (a) it’s especially important to compensate for inequalities in cash, and (b) we presume that test scores measure educational success. Both presumptions are debatable, but I believe them enough that I’ll keep attending meetings on how to measure them better.