President Obama, in each of his last three State of the Union addresses, spoke urgently of the need to cut through the "red tape" in Washington.

But regulatory costs for the American public and business community, it turns out, soared during his first term. A new report by the conservative Heritage Foundation estimates that annual regulatory costs increased during Obama's first four years by nearly $70 billion -- with more regulations in store for term two.

"While historical records are incomplete, that magnitude of regulation is likely unmatched by any administration in the nation's history," the report said.

The analysis by Heritage did not count every single regulation issued in Obama's first term, but looked at "major" regulations impacting the private sector. It came up with 131 over the past four years -- many of them environmental. In addition to the $70 billion in annual costs from those rules, the report estimated that new regulations from the first term led to roughly $12 billion in one-time "implementation costs."

The math is up for debate. Even Heritage acknowledges there is no "official accounting" for federal regulatory costs. But government agencies, as well as think tanks like Heritage, have tried to track the price tag by looking at records maintained by the Government Accountability Office and agencies' own estimates.

Heritage found the costliest regulations between 2009 and Jan. 20, 2013, came out of the Environmental Protection Agency, with their rules imposing nearly $40 billion in costs. Next in line was the Department of Transportation, followed by the Department of Energy.

The Department of Health and Human Services was in the middle of the pack, though with regulations from the federal health care overhaul still in the pipeline, costs associated with that agency could rise in the years to come.

The costliest rule was issued by both the EPA and Department of Transportation, imposing new fuel economy standards on U.S. automobiles. It's estimated to cost $10.8 billion annually, potentially adding $1,800 to the price of a new car as manufacturers spend more money to comply.

Costing nearly as much was an EPA rule requiring utilities and other fossil fuel plants to limit emissions -- though part of that rule is still under review.

Though environmental rules were the costliest, Heritage found that the highest number of regulations in 2012 were actually in the financial field as a result of the "Dodd-Frank" financial industry overhaul passed by Congress.

The Obama administration acknowledges that EPA rules are the costliest of any agency. But the administration claims those rules also come with the biggest benefits -- benefits that far outweigh the costs.

A report put out earlier this year by the White House Office of Management and Budget said many of the claimed benefits from EPA clean air regulations "are mostly attributable to the reduction in public exposure to a single air pollutant: fine particulate matter."

The EPA claims that changes made to emissions standards and other areas will save billions in health costs for the public.

Separately, a spokeswoman with the Office of Management and Budget estimated that the benefits of new rules since the start of the Obama administration are even greater.

"OMB's Draft 2013 Report to Congress on the Benefits and Costs of Federal Regulation -- which is based on agency analyses developed using the same unbiased principles that previous Administrations have used for the last 30 years -- shows that the estimated net benefits of rules finalized through the fourth fiscal year of the Administration are estimated to be $159 billion, compared to $40 billion through the fourth fiscal year of the Bush Administration," spokeswoman Ari Isaacman Astles said.

The OMB report estimated that in fiscal 2012, 14 major rules came with between $14.8 billion and $19.5 billion in annual costs, but with between $53.2 billion and $114.6 billion in annual benefits.

The Heritage report's estimate of the annual costs imposed in 2012 were not that far off -- Heritage pegged the annual cost of 2012 rules at $23.5 billion.

The Heritage report did not delve deeply into the benefits of all these regulations, though suggested the administration has exaggerated those numbers. The analysis said the "particulate matter" pollutant EPA often cites is already subject to EPA regulations, calling the claimed benefits of additional reductions "speculative."