For almost fifty years, the US government has had an especially effective tool for ensuring fair elections: sending teams of federal observers to polling stations across the country. Though relatively little known, the program has been crucial in dismantling the discriminatory practices that disenfranchised voters of color. In the program’s early days, federal monitors risked their lives to collect evidence courts needed to outlaw the electoral mechanisms of Jim Crow. And as recently as the 2012 presidential election, the Justice Department dispatched more than 780 federal employees to 51 jurisdictions across 23 states.

As a result of a 2013 Supreme Court decision, however, the program is now being quietly curtailed. In 2013, the Supreme Court hobbled the 1965 Voting Rights Act, which for decades had provided safeguards to prevent unfair voting practices, including special oversight for jurisdictions with a history of voter discrimination. In Shelby County v. Holder, the Court found that Congress created a flawed formula to select those special jurisdictions. Last week, the Justice Department revealed that, in light of the Supreme Court decision, it has concluded that the Attorney General no longer retains the statutory authority to send observers to those jurisdictions.

The decision of the Justice Department, where I worked for five years, comes as state legislatures, primarily in the South, compete to limit access to the polls by the poor and by racial minorities. North Carolina continues to pursue a campaign to cut back the number of days in which voters can cast their ballots—a step that especially affects hourly-wage workers who cannot leave work on election day. Texas seeks to revive a draconian voter-ID requirement that, according to one federal court, forces poorer citizens to choose between their livelihood and their franchise.

Until now, federal observers monitored polls in states and jurisdictions covered by the Voting Rights Act and also in other jurisdictions that consented to be monitored. Now, absent a court order, the Attorney General must request and receive permission to monitor polls in any jurisdiction. It’s hard to imagine North Carolina or Texas giving the federal government front seats to the consequences of their voter identification policies.

These jurisdictions have no legitimate reason to deny the government permission to observe the polls. Observers, well-trained federal employees drawn from across the government, do not interrupt or interfere with polling place operations. In fact, these civil servants, paid volunteers often in their fifties and sixties, work in pairs and spend election day sitting silently in the corner: they watch; they take notes; they document each voting irregularity they witness. Monitors leave the polls to interview turned-away voters and to update Justice Department lawyers.

I led federal observers in dozens of jurisdictions, and in my experience, even before the Supreme Court decision, it was often difficult for the Justice Department to monitor suspect jurisdictions not covered by the VRA. Such jurisdictions make all sorts of excuses to deny the department’s monitoring requests: federalism and states’ rights, concerns about big brother and beltway partisanship, claims that polling places smaller than a shoe box would be too crowded to manage with monitors present. One Texas election administrator told me that she worried small-town voters might feel intimidated by the presence of G-men. But, in the end, the truth is that these election administrators simply do not want to hand Justice Department lawyers eyewitness accounts of election-day misconduct. The denial, of course, is understandable: no man wants to fashion the noose that will be used to hang him.

Meanwhile, observers continue to witness shocking behavior inside the polls. In a single day in 2003, federal observers in Noxubee Mississippi documented more than 180 instances of coordinated misconduct designed to rig a local election. A year later, the department sent federal observers to Boston’s Chinatown to investigate allegations that poll workers were violating the rights of Chinese-American seniors, many of whom were functionally illiterate or vision impaired. Some poll workers, under the pretense of offering assistance, entered the voting booth with these voters, promising to mark the ballot consistent with the voters’ wishes. And yet federal observers witnessed poll workers brazenly ignoring the voter’s choices, catching corrupt poll workers casting the ballot for a candidate the voter did not choose. In 2011, a federal court relied upon observer reports to conclude that Sandoval County, New Mexico had effectively disenfranchised members of the Keres tribe. This type of first-hand evidence is nearly impossible to catch without eyes and ears inside the polls.

In fact, federal observers do more than collect evidence of election-day wrongdoing. The government uses observer reports to provide communities with essential feedback. In the years following Hurricane Katrina, for example, federal monitors regularly watched polls in New Orleans. They witnessed chaos: residents shuffling between incorrect precincts, polls running out of ballots, long lines in poorer neighborhoods—the kind of election-day failures many Americans don’t know exist. In some polling places, as many as one in five voters were turned away. Federal, state, and parish officials used monitor reports to improve New Orleans’s organization and practices. In cities like Milwaukee that have had a surge in Spanish-speaking citizens, Justice Department monitors have identified underserved precincts that require bilingual poll workers and translated ballots.

If these reasons weren’t enough, consider this: voters generally appreciate the presence of federal monitors. The department sends observers to the most broken jurisdictions, communities torn apart by discrimination and partisanship. In these communities, mistrust pervades local government. Both sides welcome outside intervention to cool tempers and to ensure the integrity of their elections.

The next federal election is four months away, and the Justice Department, for the first time in half a century, will lack the power to monitor polls in communities where monitoring is most needed. A bipartisan group of lawmakers has proposed a revised Voting Rights Act, a bill designed to cure the defects outlined by the Supreme Court. But the bill has languished, and no one expects Congress to take it up any time soon. So, in November, the federal government will have little power to guarantee a fair election, while voters will have less recourse against election-day misconduct than at any time since the Civil Rights era.