I’m following Mitt Romney’s example and vacationing in New England this week, but the Sunday-morning talk shows make it to out to Cape Cod, and I couldn’t quite go cold turkey. In addition to rehashing of Friday’s Bain Capital story, the network gabfests actually generated some news on another front: three well known Republican pundits—Bill Kristol, George Will, and Matthew Dowd—all criticized Mitt Romney for not releasing more of his tax returns.

“He should release the tax returns tomorrow: it’s crazy,” Kristol, the editor of the Weekly Standard, said on “Fox News Sunday.” “You gotta release six, eight, ten years of back tax returns. Take the hit for a day or two.” Speaking on ABC’s “This Week,” Will, the veteran columnist, agreed, saying, “If something going to come out, get it out in a hurry.” And Dowd, who was the chief strategist for the Bush-Cheney campaign in 2004, said Romney’s refusal to release returns for the years prior to 2010 was a sign of his “arrogance.”

With even prominent Republicans saying that his current stance is unsustainable, the obvious question to ask is: Why is the Mittster being so obstinate? He surely isn’t standing on principle, for what principle would that be? The notion that very rich men running for President shouldn’t have to disclose as much information about their personal finances as less wealthy candidates? The principle that if your father also ran for President, and released twelve years of tax returns, then you can release just two and claim the family average is a respectable seven years?

No. It’s only fair to assume that Mitt is doing what he always does: acting on the basis of a careful cost-benefit analysis. Will’s comments on this were spot on: “The cost of not releasing the returns are clear,” he said. “Therefore, [Romney] must have calculated that there are higher costs in releasing them.” But what information could the earlier tax returns contain that would be so damaging if it were brought out into the open? Obviously, we are entering the realm of speculation, but Romney has invited it. Here are four possibilities:

1. Extremely high levels of income. According to the tax filings and estimates he released earlier this year, Romney earned about $21.7 million in 2010 and $20.9 million in 2011—most of it from capital gains on investments related to Bain Capital. This is a lot of money, but Romney may well have earned considerably more in earlier years. His ten-year severance agreement with Bain Capital ended in 2009. The terms of it haven’t been revealed, but quite probably it allowed Romney to keep pocketing a substantial portion of the firm’s profits. And the years before 2008 were massively successful ones for Bain and other private-equity firms.

But even if Romney earned fifty million dollars a year in some years, would be that be sufficient reason for him to keep his returns secret? I doubt it. Americans don’t begrudge people a telephone-number income as long as they are perceived to have earned it. And insofar as any private-equity mogul earns the money he makes, Romney earned his. He created Bain Capital and ran it for (at least) fifteen years.

2. More offshore accounts. The Obama campaign has already made much of Romney’s Swiss bank account, his Bermuda-based investment company, and the income he receives from Bain Capital-related trusts that are domiciled in the Cayman Islands. It is perfectly possible that in the years before 2010, Romney and his financial advisers were even more aggressive in their use of overseas investment vehicles and tax shields. Hedge funds and private-equity funds, such as Bain Capital, routinely exploit offshore shell companies, to minimize their tax burdens but also to escape government regulation and oversight.

Here, too, though, I doubt whether this would be sufficient reason to justify not releasing the returns. Thanks to the Obama campaign, Romney’s name is already indubitably linked to offshore accounts. I doubt whether the average American is very shocked. Most people know full well that one of the reasons rich people employ so many fancy accountants and tax lawyers is to shield as much of their income from tax as possible, and that one of the ways they do this is by keeping cash offshore.

3. Politically explosive investments. Over the years, Bain Capital invested in all sorts of companies, including some that helped corporations outsource and offshore some of their operations, and one, a medical-waste-disposal company, that helped family-planning clinics to dispose of aborted fetuses. Romney, even after he left, kept much of his money in investment funds managed by the firm.

It is conceivable that Romney’s earlier tax returns reveal that he made a lot of money out of some controversial investments. Again, though, it is difficult to see how this would justify taking more heat on the tax issue. The Obama campaign is already attacking him mercilessly for Bain’s actions in shutting down plants, slashing wages, shifting production offshore, and driving companies into bankruptcy while extracting hefty fees from them. Unless Romney were investing in a Chinese sweatshop company that was taking business from American firms, or helping to finance a chain of for-profit abortion clinics, most of the damage has already been done.

4. A very, very low tax rate. Romney’s filings indicate that his effective federal income-tax rate in 2010 was 13.9 per cent, and his estimated rate for 2011 is 15.4 per cent. Those figures reflect the fifteen-per-cent tax rate on capital gains and dividend income. But it is perfectly possible that in earlier years he paid even lower rates. Since his 2010 and 2011 returns were prepared during an election campaign, it seems likely that his accountants took a conservative approach to deductions and other aspects of his finances. In prior years, they may well have been more aggressive. And maybe at some point Romney suffered some investment losses that enabled him to reduce his tax burden in subsequent years. Obviously, we don’t know. But there may have been a year in which Romney’s federal tax rate was in the single figures, and possibly even close to zero.

This seems pretty unlikely. But, hey, as Matthew Dowd noted, “there’s obviously something there, because if there was nothing there, he would say, ‘Have at it.’ ”

See the Political Scene for The New Yorker’s full coverage of the campaign season.

Photograph by Evan Vucci/AP Photo.