Consider this fairly recent NYT piece, titled “A Simple Way to Help Low-Income Students: Make Everyone Take SAT or ACT.” The essay talks about research that shows that making these tests mandatory in high schools raises the participation rate (duh) and in so doing identifies high-achieving students who ordinarily would not have taken the test. See, typically those who are screened out of taking a college entrance exam through self-selection are those who are less college ready and perform less well. But this is far from universal, and there are many potential high-scoring testers who are screened out through fees, lack of parental guidance, or a lack of information about when and how to sign up. The research discussed showed that low-income but high performing students are less likely to take these tests than high-income, high-performing students, and that making the tests mandatory will thus lower the relative disadvantage of those students. Mandating the test is thus a tool for equality – it increases the opportunity for students who are typically systematically excluded from college.

Let’s think about things for a second. First, and to be clear, the research does not show that low-income students are more likely to perform well. The opposite is the case:

(Data’s a bit old, but this is a durable outcome.) So having more low-income students taking the SAT will likely mean finding that many low-income students are in fact not prepared for college, to go along with finding those high-performing kids who we wouldn’t otherwise. Still, obviously I think it’s a noble and necessary goal to help identify talented students from poor families. The point is that it’s odd to think of this as a project for increasing equality as such. We’re simply looking for more “diamonds in the rough,” and hopefully helping to pull them out from their peers – who are thus left even further behind.

Here’s a point to stress: the very purpose of educational testing is to identify inequality. That is, we develop and administer tests precisely to better understand how students are not the same. In fact, the most precisely that tests are, the more unequal we understand the tested population to be. A 10 question test likely has less discriminatory power than a 100 question test, and thus the 100 question test is more likely to differentiate between closely-grouped students – that is, to identify how they are unequal. Progress in educational testing stems from designing instruments that are more sensitive to underlying inequality. That’s the very name of the game.

As I’ve said before, we talk about education as fulfilling two functions that are not just in tension with each other but directly contradictory: education is discussed as a tool for creating greater socioeconomic equality, and as a system for identifying excellence and rewarding it with status and economic opportunity. The problems here should be obvious.There’s a much larger conversation about summative equality and equality of opportunity here, which is too directly political to get into in this space. (I will say that I think equality of opportunity is not really a coherent idea when you pull at it a bit.)

But from the standpoint of educational policy, it’s not clear to me that we really know what we want to be doing. Some people tell me that our goal should just be to move everyone up in terms of absolute achievement, raising averages without necessarily changing relative performance. That might have lots of good effects, but it’s by definition not something that could help with inequality, as what’s rewarded by the labor market is relative educational achievement, not absolute. If everyone who ever went to an Ivy League school was sent to the moon, they’d simply look for the next rung down and hire accordingly. If the purpose is instead to shrink the variance, to narrow the range between the top and bottom of the achievement scale, we’d want to talk about limiting resources to the top-performing kids, and we’d still be looking for differences in what individuals can do. And we have no good reason to think that we can achieve either at scale, because while some interventions have helped different groups at different times, the general bell-shaped distribution of overall achievement on any identifiable quantitative metric of academic success has been persistent and unchanging over time.

As long as we use education as a system for sorting students into different tranches of ability, and as long as that sorting system is a key mechanism for placing people into different levels of income and joblessness, we can’t conceive of our system as being an engine of socioeconomic equality. We might sometimes use testing to identify areas where more resources are needed and distribute them accordingly. But 15 years since No Child Left Behind and the testing-heavy era it augured, we have seen almost nothing in the way of convincing proof that testing is a reliable tool for raising standards and increasing either equality or opportunity. Tests are powerful things, and modern test-development can produce exams of extraordinary precision. But they can’t be useful until we have a clear and coherent vision of what we’re testing for. To get that understanding, we have to begin to pull apart our basic assumptions about education and our economy, to ask ourselves if the system can do what we imagine it can do.