A recent New York Times analysis suggests that a generation of policies meant to bring racial proportionality to our selective colleges has failed. “Even after decades of affirmative action, black and Hispanic students are more underrepresented at the nation’s top colleges and universities than they were 35 years ago,” declared the authors.

In 2015, black and Hispanic students made up 15 and 22 percent of the U.S. college age population, respectively, but just 8 and 14 percent of the enrollments at top universities. Those gaps are wider today than in 1980—and, in the case of Hispanic students, the gap has tripled over that time.

Yet these higher-education problems are a consequence and symptom of a systemic ailment that is typically caught as soon as students of color enter the classroom. Some, of course, have been immunized before they get there and some were hit well before kindergarten. For many, however, the K–12 system is where trouble begins. “Elementary and secondary schools with large numbers of black and Hispanic students are less likely to have experienced teachers, advanced courses, high-quality instructional materials and adequate facilities,” writes the Times, citing data from the Office for Civil Rights.

These school-level deficiencies are legion and—at least vis-à-vis the vexing gaps in elite colleges—they do the greatest harm to high-ability youngsters from disadvantaged circumstances, those being the kids most likely to gain acceptance into top universities, provided their primary and secondary schools cultivate their potential.

More than middle- and upper-middle-class pupils, these children depend on the public education system to do right by them. Many lack the supports from family and community that their more advantaged peers enjoy, and many attend schools awash in low achievement, places where all the pressures on teachers and administrators are to equip weak pupils with basic skills in the three R’s. Such schools understandably invest their resources in boosting pupils up to and over the proficiency bar. They’re apt to judge teachers by their success in accomplishing that. And they’re not apt to have much to spare—energy or time, incentive or money—for students who are already achieving adequately.

Fortunately, this is a treatable malady, so long as schools and school systems grasp the urgency of creating more opportunities for black and Hispanic students who show strong academic potential. The challenge is ensuring the poor kids also have access to those opportunities, ideally in the form of special programs of various kinds (STEM, English language arts, fine arts, etc.) designed for students with high ability. As we argue in our 2015 book, Failing Our Brightest Kids: The Global Challenge of Educating High-Ability Students, the best way to maximize the education of high-ability, disadvantaged children is to start by giving everyone a fair shot at what’s available. Schools can do that by taking advantage of universal screening when they measure student achievement in core subjects, inasmuch as such measures are required, starting in third grade, under the federal Every Student Succeeds Act. Use those results to identify the top 5 percent or so of test takers in each school, not the top 5 percent in the district or state. This will diversify the qualifying population, allowing more high-achieving black and Hispanic students to take part in programs that challenge them.

Indeed, economists Laura Giuliano and David Card have found that universal screening works especially well for minority students. And looking for top performers in every school is why Texas allots 75 percent of U.T. admissions to an equal proportion of top graduates from every high school rather than the top graduates statewide.

Next, ask elementary teachers to nominate another 5 percent of their pupils, children who may not be top scorers but who show uncommon potential. Encourage them to look at everything, not just grades and test scores.

This two-step process is not common in school districts across the country. But if more employed it, they’d locate about 10 percent of kids who are already high achievers or, in their teachers’ eyes, could be. And far more disadvantaged students would have access to advanced courses, teachers trained to educate high achievers, classes with similarly able peers, and other opportunities to realize their full potential.

That’s almost a no brainer. Yet federal and state laws largely ignore high-ability poor kids, making them among the most neglected populations in American public education. Which leads, along with many other forms of wasted human capital and suppressed mobility, to the sorry findings reported by the Times.

As long as there are racial differences in income and wealth, family structure, and early childhood experiences, we’re not going to close this gap entirely. But schools aren’t doing enough to narrow it. Smart kids deserve an education that meets their needs and enhances their futures, just like children with other distinctions. They have their own legitimate claim on our conscience, our sense of fairness, our policy priorities and our education budgets. And if we cannot bring ourselves to start pushing more of these smart kids from every background as far as they can go, we’ll keep suppressing the supply of tomorrow’s inventors, entrepreneurs, artists, and scientists. And we’ll keep robbing tens of thousands of black and Hispanic students of opportunities for which they’d otherwise qualify.