In the past few months, a former police chief in Pennsylvania pleaded guilty to voter fraud in a town-council election. That fraud had flipped the outcome of a primary election. Former Connecticut legislator Christina Ayala has been indicted on 19 charges of voter fraud, including voting in districts where she didn’t reside. (She hasn’t entered a plea.) A Mississippi grand jury indicted seven individuals for voter fraud in the 2013 Hattiesburg mayoral contest, which featured voting by ineligible felons and impersonation fraud. A woman in Polk County, Tenn., was indicted on a charge of vote-buying—a practice that the local district attorney said had too long “been accepted as part of life” there.

Now come the midterm elections on Nov. 4. What is the likelihood that your vote won’t count? That your vote will, in effect, be canceled or stolen as a consequence of mistakes by election officials or fraudulent votes cast by campaign workers or ineligible voters like felons and noncitizens?

Unfortunately, we can’t know. But one thing is almost certain: Voter fraud will occur. Many states run a rickety election process, lacking rules to deter people who are looking to take advantage of the system’s porous security. And too many groups and individuals—including the NAACP, the American Civil Liberties Union and U.S. Attorney General Eric Holder—are doing everything they can to prevent states from improving the integrity of the election process.

Their refrain is that voter fraud either doesn’t exist or is so insignificant that nothing needs to be done to improve ballot security. Yet in the U.S. Supreme Court’s 2008 ruling that upheld Indiana’s voter ID law, Justice John Paul Stevens acknowledged “flagrant examples of such fraud” throughout the nation’s history and observed that “not only is the risk of voter fraud real” but also that “it could affect the outcome of a close election.”

Polling shows that the November general election will likely have many close races, particularly on the local level. Nothing new there. In 2014, 16 local races in Ohio were decided by one vote or through breaking a tie. In 2013, 35 local races in Ohio were that close.