Many Americans have come to recognize that certain systems and structures were designed to preserve white supremacy. Mass incarceration, for instance, is today widely understood as an intentional policy choice designed to lock up black people—a system that perpetuates itself by stripping blacks of their right to vote. (Alabama’s explicit justification for its felony disenfranchisement law? To “establish white supremacy in this state.”) In a recent article, I noted that, in a much less dramatic but still insidious way, the Electoral College is one of the structures that is, from the roots up, fundamentally complicit in the suppression of minority rights. This note spurred a great deal of pushback and disbelief, so it may be helpful to explain at length why the Electoral College is, historically speaking, an instrument of white supremacy.

From James Madison’s records of the Constitutional Convention, we know that the delegates disagreed vehemently about how, exactly, the executive at the head of their new government would be chosen. Many delegates’ chief concern was that the executive be fully independent from Congress so as to preserve separation of powers. This desire for executive independence spurred a majority of the delegates to reject any plan by which Congress simply appointed the president. The obvious solution to this problem was some kind of popular election, through which the (white, land-owning, male) voters could directly choose the commander in chief.

But as Madison tells it, there was “one difficulty … of a serious nature attending an immediate choice”—that is, a popular vote—“by the people.” That wrinkle? “The right of suffrage was much more diffusive in the Northern than the Southern States; and the latter could have no influence in the election on the score of the Negroes.” Madison’s meaning here is straightforward. Southern states had a huge mass of people—slaves—who contributed to population size but could not vote. Northern states did not. And the Southern delegates felt entitled to national representation equal to their total populations, slaves included—despite the fact that slaves could not choose their representatives and were not, in any real sense, actually represented at all.

By the time the delegates got around to debating presidential elections, they had already resolved the dispute over the counting of slaves by passing the notorious Three-Fifths Clause. This “compromise” counted slaves as three-fifths of a person for the purposes of apportioning representatives to the House. To make things easy, the delegates decided to use this calculus to count population for the Electoral College, adding up “the whole number of senators and representatives to which the state may be entitled in the Congress” to determine how many electors a state should receive.

Slave states received extra representatives thanks to the Three-Fifths Clause; these extra representatives translated into extra electors in presidential elections. The effect of this choice was nearly immediate: In the election of 1800, slave-owning Thomas Jefferson beat abolitionist John Adams only because slave states received extra electors. If they hadn’t, abolitionist Adams would’ve triumphed.

The Electoral College eventually saw reform—yet even after some fine-tuning, the same basic quandary kept emerging. The Civil War Amendments abolished the Three-Fifths Clause, insisting that “representatives shall be apportioned … counting the whole number of persons in each State” while prohibiting race-based voting discrimination. But the 14th Amendment explicitly reserved voting rights to “the male inhabitants” of a state, denying this newly expanded suffrage to women. As Akhil and Vikram Amar have noted, this sexist exclusion maintained a quintessential problem with the Electoral College: It gave heavily populated states outsized clout in choosing a president—while allowing those states to deny voting rights and representation to a huge portion of their populations.

Obviously, this sexism situation was ultimately remedied by the 19th Amendment. But for decades, the Electoral College allowed states to continue repressing women’s voting rights without consequence. Akhil and Vikram Amar explain that the founders’ system:

encouraged the continued disfranchisement of women. In a direct national election system, any state that gave women the vote would automatically have doubled its national clout. Under the electoral college, however, a state had no such incentive to increase the franchise; as with slaves, what mattered was how many women lived in a state, not how many were empowered.

Most of us are taught that the Electoral College was designed to dilute democracy by forcing voters to choose independent electors, who then chose a president. That is true, but it’s only part of the story. The system was also designed to accommodate and preserve slavery, the ultimate tool of white supremacy—and later served to delay universal women’s suffrage. Today, that first justification is irrelevant, since electors are generally no longer independent. But the second remains all too relevant, because in its current form, the Electoral College continues to dilute the votes of minority voters.

As I pointed out in my earlier piece, there is a relatively simple fix here, one proposed by Akhil and Vikram Amar along with legal luminary Robert W. Bennett. If states whose electoral votes add up to 270 agree to assign their electors to the winner of the popular vote, they can effectively eliminate the Electoral College. This agreement, called the National Popular Vote Interstate Compact, or NPVIC, has already passed in states with a combined 165 electoral votes. Persuade a few big swing states to join, and the NPVIC would render the Electoral College a complete anachronism.

It may be odd to imagine California’s 55 electoral votes going for, say, President Mike Pence, should he win election in 2024. When Republicans win, residents of blue states take some pride in the fact that their electoral votes go to their candidate. But this pride is really quite ridiculous and is far outweighed by the benefits of a system that follows the popular vote. States are a collection of human people, not electors, and the president they choose represents all of them. Abolishing the Electoral College would be a good way to recognize this basic fact of a modern constitutional democracy.