It is easy to dismiss the latest conservative rants against early voting as just one more way for Republicans to try to gain advantage over Democrats at the polls. But something much more troubling may also be at work: Some opponents of early voting are promoting the view that a smaller (and skewed) electorate is better for democracy.

In the past few weeks, a flurry of conservatives have attacked early voting, from Eugene Kontorovich and John McGinnis in Politico to George Will in the Washington Post to J. Christian Adams in the Washington Times. The timing is no coincidence: The Presidential Commission on Election Administration, which President Obama created to look at issues with long lines and other election problems, recently issued its much-anticipated report. The report is full of many sound suggestions for improving our elections, and one of the key recommendations is to expand early voting, either in person, through absentee ballots, or both. There’s good reason to follow the commission’s recommendation: Early voting takes pressure off administering the vote on Election Day. It helps avert long lines and aids election administrators in working out kinks. Voters like early voting because it lets them pick a convenient time to vote, when there are not work or child-care conflicts.

And yet, the commission’s stance on early voting could have been derailed by politics. The commission was co-headed by Barack Obama’s election lawyer, Bob Bauer, and Mitt Romney’s election lawyer, Ben Ginsberg, and some observers thought that Ginsberg might take heat for endorsing early voting because in recent years it has benefited Democrats. That’s why the Republican legislatures in Florida, North Carolina, and Ohio have cut back on early voting. To their credit, Ginsberg and other Republican commissioners saw the sound basis for expanding early voting and ignored the partisanship surrounding the issue.

The conservative critique against early voting, meanwhile, goes deeper than the partisan concern about boosting Democratic turnout. George Will bemoans early voting as “diffuse and inferior.” Adams says, “Early voting means stubborn voters will make uninformed decisions prematurely. Voting even one week early produces less-informed voters and dumbs down the electorate.” Kontorovich and McGinnis complain: “People will be able to vote when the mood strikes them—after seeing an inflammatory ad, for example. Voting then becomes an incoherent summing of how various individuals feel at a series of moments, not how the nation feels at a particular moment.”

Despite their opposition to early voting, Kontorovich and McGinnis don’t seem to have a problem with “old fashioned absentee ballots.” That’s nonsense. If one really believes that we need a set Election Day to all make a deliberative choice together, absentee balloting is even worse than early voting because it does not even happen in public with other voters and it raises a real risk of voter fraud—much more than in-person voting. (Will and Adams, for their part, generally don’t like absentee voting.)

All of these conservative commentators agree that everyone should vote on Election Day to promote “deliberation” or to prevent “stubborn” voters from making “uninformed” or emotional decisions “prematurely.” In short, they argue that we cannot trust the people to decide for themselves when they have enough information to vote.

The claim is empirically false. As Doug Chapin explains: “This argument, which was popular a decade ago, is undercut by research by Paul Gronke and others showing that early voters are not only more partisan but less undecided, meaning that they have no interest in ‘taking in the full back and forth of the campaign.’ It also flies in the face of voters, well, voting with their feet by choosing to cast ballots outside of the traditional polling place.”

But conservative critics of early voting runs don’t just mistrust early voters; they mistrust voters in general. As I explained here, there is a fundamental divide between liberals and conservatives about what voting is for: Conservatives see voting as about choosing the “best” candidate or “best” policies (meaning limits on who can vote, when, and how might make the most sense), and liberals see it as about the allocation of power among political equals. Cutting back on early voting fits with the conservative idea of choosing the “best” candidate by restraining voters from making supposed rash decisions, rather than relying on them to make choices consistent with their interests.

In a sense, the more barriers to voting, the better. Consider how Jonah Goldberg put it in a 2005 Los Angeles Times column: “Voting should be harder, not easier—for everybody. … If you are having an intelligent conversation with somebody, is it enriched if a mob of uninformed louts, never mind ex-cons and rapists, barges in? People who want to make voting easier are in effect saying that those who previously didn’t care or know enough about the country to vote are exactly the kind of voters this country needs now.” It is probably no coincidence that the comments on Adams’ piece on the Washington Times website go even further, by endorsing things like literacy tests or civics tests before voting. I wish the calls to cut back on early voting were merely partisan. But I fear not.