I’m back from a meeting of people who practice and advocate mixed-methods research (research that integrates quantitative and qualitative data). They have identified barriers or biases against such work. Editors and reviewers tend to be either quantitative or qualitative experts, journals impose tight word limits that are frustrating if you want to describe two complementary methods, and so on.

A more general question is how any type of scholarship gains legitimacy. I have observed several efforts to legitimize new methods, such as Community Based Participatory Research (CBPR) and Participant Action Research (PAR), as well as defenses of older methods that are being squeezed out, such as philosophical argumentation within the discipline of political science.

It is worth considering what gives scholarly methods legitimacy in the first place. I would offer a roughly Weberian theory. For Weber, “modernity” means secularization and specialization. Under those two conditions:

1. It pays to demonstrate a specialized skill or capacity, because desirable social roles are now doled out to specialists—not (or at least not officially) to people who have social rank and pedigree.
2. Specialists not only receive, but they also need, tools and methods that require scarce resources. A particle physicist needs a supercollider, to name an extreme example. If you can’t get access to the necessary instruments, you can’t practice the trade.
3. The society as a whole lacks confident, consensus beliefs about ethics, metaphysics, epistemology, and aesthetics (the classic four pillars of philosophy). But you can’t talk or think very well without having beliefs about those matters, and it’s difficult to justify them satisfactorily to people who disagree. Therefore, we make routine progress within smaller communities that share beliefs or that may even be defined by their shared beliefs.

You can see the result of these conditions in the development of academic fields. For instance, classicists used to be numerous and influential in universities. They possessed specialized skills: fluency in Greek and Latin and the experience of having already read all the major ancient classics at least once. These texts were widely believed to be better (ethically and aesthetically) than most or all of what had been written since. However, whole categories of people could not read them; for instance, very few women were taught Greek or Latin. They certainly couldn’t see the rare ancient manuscripts needed for the philological work of establishing authentic texts. Thus, being a classicist was rewarded with status and with scare resources, such as access to teaching jobs and libraries.

As the Greco-Roman classics receded in importance and lost their privileged place in the culture, people began to want to study modern literature. But virtually everyone in a country like England could read works in English. How then could English literature professors justify their special social role? One step was to develop a canon of difficult works that could claim to be as valuable as the ancient classics were. Having read Shakespeare and Milton set you apart a bit. Another step was to introduce philology and epigraphy to the study of modern texts. (That also required direct access to manuscripts and rare printed volumes). And a third step was to develop specialized and non-intuitive ways of reading, such as by applying theoretical frameworks.

Already in the eighteenth century, the editors of The Literary Magazine could claim legitimacy on the basis of specialization: “a selection has been made of men qualified for the different parts of the work, and each has the employment assigned him, which he is supposed most able to discharge” (quoted in Kramnick 2002).

At that time, there was still considerable consensus about values. In modernity, however, ethical, metaphysical, epistemological, and aesthetic values are seen as controversial and perhaps culturally relative. Fortunately, you needn’t justify a given philosophical premise in order to write an ordinary work of literary criticism today; you can just cite a major theorist who has been deemed legitimate within the scholarly community. Names of theorists become tokens that justify premises, much as scripture might justify spiritual assumptions within a traditional religion.

This is a purely external, sociological explanation of the development of modern literary criticism. I believe that the discipline yields valuable insights, so I welcome its development. Indeed, if literary criticism produced little public value, it might collapse. Specialized occupations need public support in the long run. Still, a Weberian perspective allows us to identify specialization and a reliance on canonical theorists as two responses to modernity, irrespective of whether the resulting scholarship is any good.

Most disciplines have used these means to capture scarce positions that bring status and resources.

• Many natural and some social sciences use advanced mathematical techniques, which are difficult to learn. Physics and economics enjoy relative prestige in part because they use harder math than kindred disciplines do.
• Many natural scientists need expensive instruments.
• Ethnographers seem at first to be doing what anyone can do—observing human beings in their settings. But if you have done fieldwork in an isolated village in the global South, you have bona fides to be an ethnographer instead of a layperson.
• Quantitative social science requires not only math skills but also large-n data, which is expensive to collect.
• Qualitative researchers who achieve inter-rater reliability among numerous observers have the budgets and institutional support to hire and train those observers.
• Some humanistic research requires access to rare objects.
• Some practitioners of CPBR and PAR have social capital and cultural fluency in both academia and in highly disadvantaged communities. Their ability to code-switch sets them apart.

Within these communities, certain philosophical premises are typically shared. For instance, in most of the social sciences (both qualitative and quantitative), a moral value is something that a person or group holds and that has causes and consequences. It is not something that can be shown to be right or wrong, better or worse. However, a belief about the divine is incompatible with science and thus (implicitly) false. Among theologians, obviously, both of those assumptions are widely rejected. You have to be a kind of moral relativist to speak the language of social science, but that is a minority position in philosophy and theology.

Under such conditions, an approach like mixed-methods research struggles for legitimacy. Perhaps integrating quantitative and qualitative data would yield the most reliable findings under a range of common circumstances. However, the Weberian logic of modernity encourages some researchers to maximize their specialization in math, others to maximize their specialization in ethnography; and mixed methods fall uncomfortably in between.

One solution is a Weberian judo move: as experts criticize your lack of expertise, use their momentum against them by defining what you do as a difficult new specialization. That was one tactic recommended in the conversation about mixed methods. I find it more interesting to think about ways to combat the harmful consequences of modernity in intellectual life so that we begin to assign legitimacy differently. Obviously, the ideal way would be to reward solutions to public (including cultural) problems, rather than academic methods for their own sake. But that is a hard shift to accomplish.

[Citing Jonathan Brody Kramnick, “Literary Criticism Among the Disciplines,” Eighteenth-Century Studies, Volume 35, Number 3, Spring 2002, pp. 343-360. See also the future of classics and why social scientists should pay attention to metaphysics.]