As many as 34 million Americans with disabilities could head to the polls in November, even as advocates say problems with access remain widespread. People with developmental and mental disabilities can face an especially tough time at the polls — assuming they’re allowed to vote at all.

According to the Bazelon Center for Mental Health Law, an organization that looks at voting rights for people with mental disabilities across the country, all but 11 states have laws on the books that can restrict voting for people with mental disabilities. At particular issue are people placed under guardianship (known in some states as “conservatorship”) since advocates say guardianship orders rarely have anything to do with whether a person is competent enough to vote.

Disability rights advocates argue these restrictions run contra to the Americans with Disabilities Act, which says that “reasonable” accommodations must be made so that a person can engage with the democratic process.

“We are the last demographic within the U.S. where you can take away our right to vote because of our identity,” said Michelle Bishop, a disability voting rights specialist who works for the National Disability Rights Network, an advocacy organization based in Washington, D.C. “It’s horrifying that this is happening. The idea that you can’t make a decision on who to vote for is a basic misunderstanding of what it means to be disabled.”

A court appoints a guardian to manage someone else’s affairs usually when a judge determines that the person is hampered in making their own decisions or in communicating a decision. Having a disability doesn’t necessarily equate needing a guardian, but adult Americans can and are placed under guardianship for a multitude of reasons, from being unable to manage finances to needing help with medical issues.

Comprehensive national data on the number of adults under guardianship is lacking — just 14 states report guardianship filings. A 2008 estimate from the National Center for State Courts put the number of pending guardianship cases nationwide at between 1 million and 3 million.

It is also hard to know how many people who are under guardianship or who have been deemed mentally incompetent have lost their right to vote. The seven states that automatically bar people under guardianship from voting — Alabama, Louisiana, Missouri, South Carolina, South Dakota, Tennessee, and Utah — told us they do not maintain specific records.

But the available data and our conversations with researchers and advocates indicates it’s likely in the thousands. The federal Election Assistance Commission receives information about citizens who have been removed from state voter registration rolls in every county, in every state, across the country. Its most recent survey logged 7,345 people in 28 states who had their voting rights taken away because of a finding of mental incompetence between October 2012 and October 2014. (Sixteen states either didn’t respond to the commission’s information requests or chose not to provide any data.)

Disability rights advocates say the lack of available information on guardianship and voter access means these people run the risk of being made invisible. Americans needing a guardian are often elderly — and in many cases, vulnerable — people whose disabilities are not always fully understood by the overburdened court systems that have to quickly assess them. In some cases, courts can perceive physical disability to signal mental incapacity, and while courts are supposed to investigate guardianship cases prior to a guardian or conservator being appointed, the level of investigation can vary court to court and state to state.

Advocates also say a guardianship hearing shouldn’t be the forum to decide whether someone can or cannot vote — period. “In guardianship cases, we’re finding that people are losing their right to vote when it’s nothing to do with what the court is addressing,” said Samantha Crane, the legal director and director of public policy at the Autistic Self Advocacy Network, a nonprofit organization focused on the rights of autistic Americans.

Lewis Bossing, a senior staff attorney at the Bazelon Center, thinks we should retire the traditional idea of guardianship altogether in favor of “supported decision-making” to enhance self-determination and communication for people with disabilities. “What we’re actually doing is holding people with disabilities to a higher standard,” he said.

As it stands, Bossing is concerned that people may be falling into the category of “incompetent” when their disability isn’t fully understood. “I don’t know that those rulings are made with a full evidentiary hearing or showing,” he said, adding that the process can be overwhelming for those engaging in it. “Advocates may not know how to work the system so that they can prevail in a legal proceeding like this.”

Bossing’s fears were realized in a 2012 California case involving an 18-year-old man named Stephen Lopate. Lopate is severely autistic, and he lost his right to vote when his mother became his legal guardian. Neither Lopate nor his mother, Teresa Thompson, realized this would be the case, and Thompson joined her son in his pursuit to get his vote back. As a result of Lopate’s highly publicized case, advocacy groups put pressure on the legislature to protect the rights of Californians with disabilities.

This January, the governor signed a law affirming that any person with a disability who wants to vote should be given “reasonable accommodation” to do so, in accordance with the ADA. California joined 11 other states with no restrictions on voting for people with mental disabilities, according to the Bazelon Center.

Even though the new law is a boon for Californians with disabilities, not every person in a situation like Lopate’s knows they can now petition to have their vote reinstated. Tom Coleman, a disability rights lawyer who helped Lopate in his case, filed a complaint with the Department of Justice against California in August to pressure the state into speeding up the voting reinstatement process and to ensure that people are aware of the change in the law. After Coleman’s filing, San Diego County made good on that score, issuing a new pamphlet about ballot and voter information to more than 1.5 million registered voters.

Americans with disabilities “shouldn’t have the burden put on them to reach out to the system. The system should reach out to them,” Coleman said. “Remember, the system took the right to vote away in violation of federal law; each one of these people is a victim of disability discrimination perpetrated by judges throughout the state of California.”

Coleman thinks other states would be wise to follow California’s lead. Until they do so, many Americans who should be able to vote won’t be casting a ballot this November.

Taylor Dolven and Kristin Fraser contributed reporting and research.

Illustrations by Abbey Lossing