I disagree with my friend Robert Pondiscio about many policies, but I agree with an essential aspect of his argument in a US News article entitled “Asking the Wrong Questions on School Choice.”

Some impressive-looking recent studies assert that school choice doesn’t “work” because test scores go down when families use vouchers to transfer their kids to private schools. The New York Times describes these results as “dismal” and cites “harms” to the children from voucher experiments. Pondiscio believes that the preponderance of the evidence still “tends to favor school choice.” I don’t know if that is right, but I’m with Pondiscio that “this entire debate puts the cart before the horse.”

After all, who said that test scores are the purpose of schooling? “Choice exists to allow parents to educate their children in accordance with their own needs, desires and values.” Some parents will prefer schools that achieve lower scores on state standardized tests because they value other ends. To measure choice in terms of test scores “says, in effect, that one’s values, aspirations and priorities for one’s child amount to nothing.” He concludes:

If we limit the frame of this debate to academic outputs alone, every new study provides ammunition, but never a conclusion. The real debate we should be having is, “What kind of system do we want?” Answer that question first, then use evidence to improve the school designs, policies and programs we have agreed deserve public support.

I’ve made similar points in debates about, for example, the economic impact of civic engagement, the educational value of service learning, and the evaluation of policy more generally. We live in an age when science has enormous prestige and values are widely viewed as merely subjective. In that context, it’s common to treat programs and policies much as scientists would view natural phenomena, as things that arise on their own and have measurable effects. To decide whether they “work” means measuring their causal impact with apparently objective measures, of which state-written tests are good examples. Negative results lead to strongly value-laden conclusions–such as the Times’ headline about “dismal failures”–that are not themselves scientific.

In fact, programs and policies arise because people have value commitments. They strive to make their ideas work, albeit for various and contested purposes. Usually nothing works at first. If its impact improves, it’s because people have made it work by refining and improving the practice. The reason that data provides “ammunition, but never a conclusion” is that any empirical result can be changed by revising and improving the program under study. Therefore, the fundamental question is not what works but what we should value and try to make work.

An everyday example is public schooling. Nineteenth century reformers like Horace Mann drew from previous thinkers—and from successful experiences in countries like Prussia—to propose a new idea: every child should be educated at the government’s expense in a state-funded common school under local political control. Since then, not only have educators and policymakers refined and revised most aspects of public schooling, but scholars have critically evaluated actual schools from a wide variety of perspectives. A few observers have concluded that Horace Mann’s core idea was misplaced, but most see their role as helping to make his vision become successful. Public schools did not arise like a new species in Darwinian evolution to survive or fail on its own. Nor did Horace Mann propose a hypothesis that could be tested with a single experiment (e.g., “Common schools will work.”). Rather, universal public schooling originated with an argument that combined values and empirical predictions, and it launched a process of improvement that has combined research with practice.

The question is whether individual family choice is a transcendent value. Robert Pondiscio argues that using test scores to assess vouchers makes family preferences “amount to nothing.” I think we all value parental choice to a degree, but everyone also weighs other concerns: children’s rights, interests, and preferences, community values, prosperity at the state and national scales, democratic ideals of collective self-governance, our responsibilities to children other than one’s own, and criteria of excellence that may be unpopular.

I agree that the American ideal of the common school has always been a bit problematic, and versions of family choice common in Western Europe may get the balance better than we do with our odd mix of local monopoly schools plus radical economic inequality. But if we seriously considered the question “What kind of system do we want?”, I’m not sure the answer would be a system driven by parental preferences.

I should add that another kind of reasoning (besides scientism) motivates experiments with school choice. Some people think that market competition always or generally increases efficiency, so that introducing choice into a former monopoly will improve outcomes, almost regardless of how they’re measured. Then the effects of vouchers on test scores becomes an experiment in the efficiency hypothesis. Even if the results are mixed, the existence of major, high-quality, and recent studies in which the effects are strongly negative sounds like counter-evidence. I think that should embarrass proponents of market efficiency, of whom one was Elizabeth Warren in The Two-Income Trap: Why Middle-Class Parents are Going Broke (2003):

However, as Pondiscio argues, you can support choice as an inherent value, not because of an empirical hypothesis about economic efficiency.