Every year, millions of frustrated last-minute tax filers discover that the U.S. government makes it astoundingly difficult to give it money.

Yet this is no accident. As a technical matter, it would be relatively simple for the Internal Revenue Service to turn income-tax payments into a quick and basically automatic process for the vast majority of Americans. It would cost the government a bit of cash upfront but would save citizens tons of time, hassle, and money spent on tax preparation. And there’s the problem: One man’s wasteful expenditure on tax preparation is another man’s income. The entire industry of mass-market tax preparers—Intuit, H&R Block, Jackson-Hewitt, etc.—doesn’t want the government to help you out. Even worse, they’ve been joined in their crusade by conservative anti-tax activists who’ve decided, without any real evidence, that the best way to shrink the state is to make paying taxes as annoying as possible.

How would the IRS do your taxes for you? It would work in much the way that most jurisdictions collect property taxes. The tax authority would tell you how much it thinks you owe in taxes, and you’d write a check. The main difference would be that thanks to tax withholding, many people overpay their income taxes and would receive a check from the government rather than write one. If you looked at your tax bill and thought you were being overcharged, there’d be a dispute process through which you could plead your case. For the majority of Americans who have regular jobs and who take the standard deduction rather than itemizing, that would be it most years. And, it’s important to note, anyone who wanted to could always do their own taxes rather than have the IRS do them.

Many people react to this proposal by saying they wouldn’t trust the tax man to do their taxes for them. But in fact, you already do. The way it works now is that absent you filing a protest, you end up paying what your W-2 says you should—which is what the IRS thinks you pay.

My taxes are more complicated than most people’s because I have dribs and drabs of income from a freelance piece here or a speaking engagement there. Consequently, each spring I get a bunch of 1099-MISC forms in the mail and squirrel them away until tax time. Then I open the envelopes and start entering the data. On not a few occasions—especially when I lived with a bunch of roommates and the mail situation was chaotic—it turned out months later that I missed a form somewhere and underreported my income by a few hundred dollars and thus underpaid my taxes. Guess what? The IRS always caught me. Because the IRS already knows how much income you should be reporting since the people who pay you also submit paperwork to the IRS. If I thought they were wrong, I could dispute it. But in my experience they’re always right and I’ve always screwed up. So I pay—with penalties.

Automatic tax preparation simply takes out the middleman, where you—the taxpayer—burn some time and money wrangling with TurboTax.

Now of course the IRS doesn’t know everything. Maybe you had a baby last year or got divorced. Every year, a significant minority of people are going to have to offer an update about life events. But counting up how many babies you had last year is simple, and people (hopefully) don’t get married and divorced too often. For the two-thirds of the public that takes the standard deduction rather than itemizing, that should be it. The technology exists. In fact it is used in California, where it has drastically reduced the rate of errors in tax returns.

So why don’t we use this automatic tax filing everywhere? In part, lobbying. ProPublica’s Liz Day wrote in March about how much Intuit, the maker of TurboTax, spends on blocking this kind of initiative. But lobbying alone rarely explains anything. Intuit fought against ReadyReturn in California but hasn’t prevailed. The difference between California and the U.S. Congress is that California is much more liberal. Conservative activists like Grover Norquist have gone all-in to keep the federal tax process annoying. You can see the logic: If you think taxes are bad, then paying taxes should be annoying to maintain the viability of anti-tax politics. But think harder and this looks nuts. Nobody likes filing their taxes, but you can think of lots of arbitrary ways to make it harder. We could ban tax-prep software. We could demand that the money be paid by shipping boxes full of nickels. The forms could be written in Chinese. But why do that?

The fact is that not everything in politics is a zero-sum game. Liberals think the government needs tax revenue to fund useful programs, while conservatives worry that taxes burden the private economy. Making the tax collection process less burdensome is just a win. There’s plenty of room for debate about what the legitimate scope of governments functions is, but collecting taxes is up there with national defense as one of the things that clearly has to be done. And it should be done as well as possible. For taxes, that means the majority of us should be able to take advantage of automatic filing and leave the accountants and tax-prep software for the minority of citizens who have hideously complicated finances or who are seeking obscure loopholes to exploit.