Voting rights has become an increasingly partisan issue. In Wisconsin, new voter ID laws led to brutal lines at the polls in urban areas—a development designed, even according to Republicans themselves, to suppress Democratic turnout. In Virginia at the end of April, governor Terry McAuliffe re-enfranchised all felons who had finished parole. In theory, the move returned the vote to 200,000 people. This was a refutation of a policy originally designed to explicitly deny black people the vote. It was also, potentially, a way to give more votes to more minority and poor voters, and tip a narrowly balanced purple state more Democratic in the US presidential election.

The focus on voter IDs and felon disenfranchisement—while important—has inadvertently obscured other voting rights issues. Every year, with little comment, the United States denies millions of people representation in the District of Columbia, Puerto Rico, Guam, and other territories. Washington, DC, has a population of over 650,000 people. That makes it larger than the states of Vermont or Wyoming, and yet it has no voting representatives in either the Senate or the House of Representatives. Puerto Rico has a population of around 3.5 million people, which makes it more populous than states like Nevada, Iowa, and Arkansas. But not only do Puerto Ricans lack Congressional representation, they also cannot vote in presidential elections (unlike residents of DC, who are entitled to three votes in the Electoral College).

Puerto Rico is more populous than states like Nevada, Iowa, and Arkansas.

“The lack of voting rights in DC and other territories is not quite the same as voter suppression, in that those places have never had voting rights,” says political scientist Celeste Montoya of the University of Colorado at Boulder. Nonetheless, the effects of the voting limitations are parallel; voter ID and felon disenfranchisement restrict the power of minority voters at the polls, and so does the disenfranchisement of people in DC and Puerto Rico.

Washington, DC, has historically been majority African-American; in recent years, an influx of white residents has changed that, but black people still account for 49.5% of the electorate. Puerto Rico’s population is 98% Hispanic. Both territories consistently vote Democratic in local elections. According to Montoya, “The racial voting gaps present in the last several elections make Republicans less inclined to entertain and more likely to outright block any efforts to enfranchise DC or Puerto Rico voters.”

In contrast, you’d think Democrats would be eager to enfranchise DC and Puerto Rico. Democrats have certainly shown concern for the voting rights of other minority voters. The New York Times’s election analyst Nate Cohn estimated that felon enfranchisement in Virginia could increase Democratic margins in the state by .5 points. A recent study shows that voter ID laws cause disproportionate suppression of minority voters, creating a gap of 8.5 points between black and white voters, almost double the gap between black and white voter participation without voter ID. When minority voters go to the polls, Democrats benefit; when they don’t, Republicans benefit. Even a few points around the edges can tip election outcomes.

“The racial voting gaps present in the last several elections make Republicans less inclined to enfranchise DC or Puerto Rico voters.”

But Puerto Rico and DC aren’t a few points around the edges. Millions of voters live in these places, and their enfranchisement would immediately and profoundly affect the balance of Congress. Were DC and Puerto Rico to gain statehood, they would each have two senators. Four more Democratic senators in Congress would be a massive change in the balance of that chamber. In the upcoming election, an uncertain, razor-thin contest could become much more certain.

The change in the House would be less dramatic, but four representatives (one from DC, three from Puerto Rico) would still represent an important addition. It might even put the House in reach for the Democrats in 2016.

There’s also a strong moral case for enfranchising territories. As American University law professor and winner of Maryland’s 8th district Congressional primary, Jamin Raskin, writes, there is “a haughty Congressional indifference to the priorities of the [DC] population.” National legislators have imposed anti-abortion and anti-gay fiats on DC. The 2013 government shutdown caused widespread economic hardship for federal workers in the District who had almost no democratic input into their own fate.

With no representatives, Puerto Rico is at the whim of an unaccountable Congress.

Puerto Rico, for its part, is facing a terrifying financial crisis. It may default on some of its debts; electricity and water services may be interrupted for many on the island. The crisis is the direct result of Congressional decisions that ended business tax incentives and have limited the ability of public companies to declare bankruptcy. With no representatives, Puerto Rico is at the whim of a feckless, unaccountable Congress, which has used its power and indifference to wreck the economy.

The local and national incentives should be a potent combination. But in fact, when Democrats talk about expanding voting rights, they barely mention DC or Puerto Rico, much less smaller territories like Guam or American Samoa. In part, this could be the result of the disenfranchisement itself; when entire polities are marginalized, it becomes difficult to make their concerns salient for the mainstream. Rankin also suggests that senators and members of the House—on either side of the aisle—may be loathe to dilute their own personal influence by adding to the ranks of Congress.

When entire polities are marginalized, it becomes difficult to make their concerns salient for the mainstream.

Statehood may also seem like an impossible task. Mariely Lopez-Santana, a professor of politics at George Mason University in Virginia, told Quartz that Puerto Ricans do not necessarily support statehood themselves. Statehood requires a local vote first, so that gives Congress both a reason and an excuse not to act. For its part, the District of Columbia’s population and politicians have embraced a push for statehood but they’ve managed to get little traction.

Faced with continued intransigence, Raskin has argued for a broad voting rights amendment to the Constitution, one which would enshrine the right to vote throughout the nation, striking down felon disenfranchisement, restrictive voter ID laws, and territorial disenfranchisement in one blow. But how will Democrats drum up the two-thirds majority necessary to pass such an amendment when they can’t even muster a majority for DC statehood?

The Democrats are, de facto, the party whose constituents are most likely to be disenfranchised, in numerous ways. The party has both a moral and a partisan obligation to work for their voters. That means, first of all, ensuring that those voters can vote.

It took Democrats decades to enact health insurance, which was only passed when a Democratically controlled Congress and presidency aligned. It could easily take decades to pass voting rights legislation too. For the clock to start ticking, however, Democrats have to declare enfranchisement of everyone a central priority.

They should do so soon. One party, at least, needs to embrace the rights of all voters, whether in Virginia, Wisconsin, DC, or Puerto Rico.