President Trump’s newly installed agency heads are starting to take a lead role unraveling a web of Obama-era regulations, acting alongside congressional Republicans and the president himself to roll back rules they claim hurt business or simply go too far.

Interior Secretary Ryan Zinke was the latest to peel back red tape.

On his first day of work, for which he arrived Teddy Roosevelt-style on horseback, Zinke ended a ban on lead bullets and fishing tackle on federal lands and water. The ban was imposed to protect animals from lead poisoning, but had been criticized by the National Rifle Association as an attack on gun owners.

Zinke said in a statement he determined the original order was “not mandated by any existing statutory or regulatory requirement.” The NRA thanked the new secretary for “eliminating this arbitrary attack.”

Zinke also hinted at more to come in another order, directing agencies to identify areas where recreation and fishing can be expanded.

Meanwhile, the EPA reportedly is set to reverse an Obama-era decision to lock in strict gas mileage requirements for cars and light trucks through 2025.

Together, the moves are part of a three-pronged attack on regulations issued over the last several months and years. It's what White House Chief Strategist Steve Bannon, at CPAC, dubbed the “deconstruction of the administrative state.”

The Republican-controlled Congress has moved since the start of the session to nix rules issued toward the end of the last administration. And Trump has directed others to be rolled back, a plan his agencies also are implementing.

In February, for instance, Trump signed an order instructing the Labor Department to delay implementing a rule requiring certain financial professionals to put their clients' interests first. The department could simply abolish it. Trump also ordered agencies to ease the “regulatory burdens” of ObamaCare, and look at removing two regulations for every new one.

Yet, as the final members of Trump’s Cabinet are being confirmed, incoming agency heads also appear to be acting on their own.

The EPA, under Scott Pruitt, last week withdrew its request that owners and operators in the oil and natural gas industry provide information on equipment and emissions at existing operations.

The Washington Examiner reported Monday that Trump also is planning on signing an executive order rolling back Obama’s Clean Power Plan – which requires states to cut greenhouse gas emissions by a third – as well as the Interior Department’s moratorium on coal leases.

However, the Clean Power Plan order would merely instruct the EPA to overturn it. A similar order was sent out last week, instructing regulators to re-examine President Obama’s Clean Water Rule.

In another example of agencies taking the lead, Health Secretary Tom Price says his department will go through existing health care regulations and try to "get rid" of those they determine hurt patients, as Republicans push an ObamaCare replacement bill.

Conservatives, however, are hoping the Trump administration will be an opportunity not just to roll back regulations, but get agencies out of the habit of passing their own.

“Regardless of which party controls the White House, we need to get a handle on the regulatory state. Yes, roll back what we can, but also to make sure we’re going through Congress to put checks in place to restore Article 1 [of the Constitution],” Jason Pye, director of public policy and legislative affairs for FreedomWorks, told Fox News.

EPA Administrator Pruitt holds a similar view, telling The Wall Street Journal that his job is not about increasing or reducing regulation.

“There is no reason why EPA’s role should ebb or flow based on a particular administration, or a particular administrator,” he said in a Feb. 17 interview. “Agencies exist to administer the law. Congress passes statutes, and those statutes are very clear on the job EPA has to do.”

As for revoking rules via Congress, conservatives have pointed to the Congressional Review Act – a little-known 1996 law that gives Congress 60 legislative days to reconsider any new regulations. If a resolution of disapproval is signed by the president, then the agency cannot re-submit a regulation in substantially the same form, unless approved by Congress.

The House passed a bill in January – the Midnight Rules Relief Act – that, if signed by President Trump, would allow Congress to disapprove of multiple regulations at once.

Some Republicans are suggesting a slash-and-burn approach. North Carolina Rep. Mark Meadows posted online a “100 days” list of rules he wants to see revoked.

But Pye warns most lawmakers are unlikely to be so aggressive.

“I think they’re going to be thoughtful. Some, like the Clean Power Plan or the fiduciary rule, are unavoidable -- you have to start rolling those back,” he said. “With that said, we should be pursuing legislative measures, not just rolling regulations back, but making sure a future president can’t impact negatively impact [the] economy through [regulation].”