Category Archives: education policy

the monumental task that confronts a high-stakes testing state

Let’s say you don’t especially trust teachers to assess their own students, because their ratings can be inconsistent and biased. So you want to use validated and standardized assessments to evaluate students, schools, and teachers. Let’s say, furthermore, that your state authorizes about 4,000 different courses, from kindergarten through 12th grade. (A subject like science in 3rd grade counts as a “course,” by the way.) Each course encompasses many different content areas; for instance, an American history course covers the Revolution, the Civil War, civil rights, and so on. For each topic in each course, you need assessment “items” (questions or prompts of various kinds). You need more than a few items for each topic; one question does not yield a valid score. You can’t repeat items without allowing kids to cheat by looking at old tests. And you will be testing frequently–more than once per year in each course if you consider the need for make-up tests and practice tests.

The upshot is that you will need at least several hundred thousand assessment items to make the whole system work. See Florida’s Race to the Top Assessments page for some of the documents on which my estimate is based. Thus …

  1. You face an expensive undertaking, and if you skimp, you will get poor items, written by people who are not sophisticated about the content or well trained in writing assessments. Pilot-testing items costs even more money.
  2. Even if you spend enough money, writing several hundred thousand items is a human enterprise. Error is inevitable. Some proportion of your items will be flatly incorrect or invalid in other ways. Many will be too easy or too hard, or inadequate to assess the desired skills and knowledge.

On its own, this is not an argument against high-stakes testing. The best argument in favor is that measuring pretty well is better than not measuring at all. But the cost and frailty of the whole system must certainly be taken into consideration. After all, the power of the state stands behind these assessments. If a kid cannot move on to 8th grade, or if a teacher loses his job because of test scores, that is a state decision. I think people may reasonably view it as almost a juridical process.

In the corporate context, employers are always assessing employees, and vice-versa. It is not OK if an employer’s assessments are biased or arbitrary, but using standardized measures may at least reduce inevitable bias, and the market does offer a theoretical solution to injustice (the employee finds a different job). In contrast, if a state moves from not making high-stakes assessments at all to doing so badly, it’s like imposing a new juridical regime that makes arbitrary decisions. I see a serious threat to justice.

from the achievement gap to empowerment

Harry Boyte argues that the “achievement gap” is the wrong framework for thinking about education. It “assumes the point is upward mobility — how to give poor people, especially racial minorities, resources and remediation so that they can make it in a hypercompetitive, individualist, meritocratic educational system and society.” But Harry asks, “What if the problem is the hyper-competitive, individualist education system itself, now largely a screening mechanism for personal advancement?”

I would elaborate as follows. The main policy debate about education is about individual human capital in a competitive global market. The premise is that individuals need skills to compete. Poor individuals especially need better investment to give them a leg up in the labor market. Communities’ welfare depends on the aggregate of individuals’ human capital.

The limitations of that narrative are at least as follows:

  • It omits non-cognitive skills that also redound to the benefit of the individual, such as being able to work in teams (and especially in diverse teams)
  • It overlooks social capital and collective agency as the basis of economic success for communities and nations. If we have a lot of people who know math but don’t ever work together, we will not prosper.
  • It has an implausible motivational theory: teachers, kids, and families must do whatever it takes to boost individual human capital. In reality, activities like service and politics can be more motivating.
  • It overlooks the importance of communities in education. James Coleman started the whole social capital research agenda by arguing that communities’  engagement with schools was a precondition of their success, even when success was measured in individual economic terms. It is very hard to engage communities in schools if they turn into machines for developing the individual labor market advantage of students. (Why should I engage with the school in my community if its job is to prepare each individual student to compete against my kids in the labor market?)
  • It overlooks other educative assets and resources, beyond schools.
  • It works best for the institutions that are drawing the most economically competitive students and faculty. The dominant narrative works fine for Stanford. But what is a local college supposed to do? Should it try to claw its way up the competitive rankings in search for students who already perform better before they enroll?
  • It overlooks democratic and civic outcomes, such as participating effectively in the democracy.

intelligence is more like confidence than height

(Washington, DC) According to experimental studies collected by Gregory M. Walton*:

  • You can explain to seventh-graders that people build their intelligence by working and learning, and their grades on math will be better for the whole school year.
  • You can give 7th graders a writing exercise designed to affirm their personal values, and as a result, the Black students will be half as likely to receive a D or an F on the whole course (even though their teacher is blind to whether they did the exercise).
  • You can give fifth graders a test and tell them they scored well because they are smart, they scored well because they worked hard, or just give them their scores. Then give them a very hard test on which they all score poorly. Then give them a third test. Those who were told that they scored high the first time because they were smart will do 30 percent worse on the final assessment–their confidence in their innate ability shaken.
  • You can tell African American college students that items from the GRE are just a puzzle and they will perform as well as white students, but tell them the same items are an intelligence test, and they will score much worse.

As Walton argues, your height won’t change depending on the context, but your score on tests will. That implies that intelligence is–to a significant if not complete degree–relational. It is a measure of how you relate to the immediate environment and the other people in it. Much as I would be far more confident, motivated, secure, and competent in my own living room than on a sound stage, I will be more “intelligent” in some settings than others.

These studies have profound implications for how we should test aptitude, whom we ought to promote and admit in school, college, and work, and how we should design educational institutions.

*Gregory M. Walton, “The Myth of Intelligence: Smartness Isn’t Like Height,” in Danielle Allen and Rob Reich, eds., Education, Justice & Democracy (University of Chicago Press, 2013), pp. 155-172.

where is the public on Common Core?

(Durham, NC) The Common Core standards are the most significant policy change in US education today, and they are increasingly controversial. Strong critics can be found on both the right and left. Meanwhile, the most influential proponents don’t exactly have fired-up grassroots supporters behind them. The main champions include several big foundations, Democratic and moderate Republican governors, and the Obama Administration’s Department of Education.

When read summaries of the Common Core, majorities of Americans and Californians do support it. But of course, how it is summarized will be highly influential if people are not very well informed about its content and origins. If the Common Core is described with hostility, people may be easily persuaded to oppose it. And plenty of political actors have motivations to present it critically–scoring some valid points along the way.

My own feelings are largely favorable, because I like the content of the Common Core, although I would acknowledge that everything depends on the tests that are now being developed to assess it. But even those of us who basically support the Common Core should try to understand what the public is thinking about it so that we can address their concerns.

Public Agenda’s Jean Johnson offers several reasons for the lukewarm level of public support.

First, parents are not primarily worried about low standards or inadequate academic achievement. Public Agenda has frequently found that parents put behavioral and social problems at the top of their lists of concerns. The public’s premise is that kids need the same kinds of educational content that their parents had, but they behave worse. Children either need more surveillance and tougher sanctions or else more care and attention (or, possibly, both). Either way, raising the academic bar is not a relevant solution.

I am not sure the parents are right. One of the striking developments of the last 20 years has been a decline in rates of teen crime and drug use. Kids are behaving better but face increasingly steep global competition. That is the expert’s framework, and it is based on data. But, as Johnson notes, many parents think that school is already hard enough and that testing is too pervasive and consequential.

I have no interest in trying to sell anyone on the Common Core, but would want to present it in a way that gets a fair hearing. To that end, I would avoid talking about tougher standards. After all, how “tough” they are will depend on the tests, which are still being designed. Instead, I would emphasize that they are simpler than traditional standards, easier to read and understand (and thus more fair), and less political because they are not negotiated by state legislatures and education agencies. They also give more flexibility to schools and teachers to decide what to assign.

discrimination and educational ambition

I thought this was a crucial moment in the recent debate between Jonathan Chait and Ta-Nehesi Coats:

Chait: The argument is that structural conditions shape culture, and culture, in turn, can take on a life of its own independent of the forces that created it. It would be bizarre to imagine that centuries of slavery, followed by systematic terrorism, segregation, discrimination, a legacy wealth gap, and so on did not leave a cultural residue that itself became an impediment to success.

Coats: What about the idea that white supremacy necessarily “bred a cultural residue that itself became an impediment to success”? Chait believes that it’s “bizarre” to think otherwise. I think it’s bizarre that he doesn’t bother to see if his argument is actually true. Oppression might well produce a culture of failure. It might also produce a warrior spirit and a deep commitment to attaining the very things which had been so often withheld from you. There is no need for theorizing. The answers are knowable.

Indeed, here is some pertinent evidence, courtesy of Angel L. Harris.* She He analyzes a very large survey of Maryland families, black and white, to investigate the connections among race, discrimination, and commitments to education. The results are important for anyone interested in debates about the (alleged) culture of poverty, cultural determinants of success, and racial achievement gaps.

Harris finds that black parents are more likely than white parents to think that succeeding in school is crucial to their children’s success. Blacks also place more importance than whites do on academics; they report spending slightly more time on educational activities; and they are more likely to seek academic help.

African American parents are more likely to believe that they and their children are subject to racial discrimination, although some white parents also see themselves as discriminated against on the basis of their race. The more that black parents perceive that their families are subject to discrimination, the more they see educational success as crucial for their children. But the more that white parents see themselves as discriminated against, the less they believe in education.

These results support the thesis that for African Americans, a perception of ongoing discrimination is a motivation for struggle and uplift. It is evidence of Coats’ “warrior spirit” and “deep commitment to attaining the very things which had been so often withheld from you.” But for some whites, a perception that they are being treated unfairly is a reason not to focus on education. Or perhaps they are not succeeding educationally and need an excuse that blames other people.

Given those findings, you might predict that black students would be achieve more educational success than comparable white students. In fact, when white and black students of similar socioeconomic and family backgrounds are carefully compared, blacks are sometimes found to have higher graduation rates and more years of education. (The bottom line of the second cited study: “African American men and women achieve greater years of education than white men and women, respectively, raised in identical family environments.”) But there remain stark aggregate differences in high school and college graduation rates by race. If those cannot be attributed to differences in educational ambition, then the remaining explanations would seem to include: subtler disconnections between the dominant culture of schools and those of some African American families; unequal resources outside the school (including lower numbers of committed parents and other adults per child); unequal quality of schools; unequal treatment within the same schools; and discrimination in admissions and labor markets. I think all of those factors have been demonstrated.

*Angel L. Harris, “Can Members of Marginalized Groups Remain Invested in Schooling? An Assessment from the United States and the United Kingdom.” In Danielle Allen and Rob Reich (eds.), Education, Justice, and Democracy (University of Chicago Press, 2013), pp 101-132