Vermont Sen. Patrick Leahy, chairman of the Senate Judiciary Committee, said Sunday the upper chamber will continue working on legislation to limit NSA spying, suggesting President Obama has not gone far enough in making changes to protect Americans’ privacy.

“There’s a concern that we have gone too much into Americans’ privacy,” the Democratic lawmaker told “Fox News Sunday.” “There’s still going to be legislation on this.”

Leahy said several times that congressional Democrats and Republicans both share the concern and suggested the direction of the legislation will be impacted by what Attorney General Eric Holder says when he testifies on Capitol Hill on Jan. 29 -- the day after the president’s State of the Union address.

The president announced the changes Friday in a major policy speech at the Justice Department, following a series of revelations by former National Security Agency contractor Edward Snowden that started last summer about the extent of agency spying.

Leahy said he wouldn’t fight the president on his proposed NSA changes -- including additional court approval, a non-government agency holding phone meta-data and limiting the extent of the data collection.

“I think we have a way we can do this,” he said. “I believe in going after the bad guys. But I also believe in some checks and balances, so you don’t have a government run amok.”

Leahy was joined on the show by retired Gen. Michael Hayden, a former Bush administration NSA and CIA director.

Much of the NSA spying program started in 2006 under that administration.

“There was an awful lot I liked about the [Obama] speech,” Hayden said.

However, he expresses some concern, particularly about the president allowing the NSA to follow only the second call from an original call or “hop,” not the third call.

“If the third hop hadn’t been useful, we wouldn’t have been using it,” Hayden said.

Holder will testify before the Senate Judiciary Committee. Obama on Friday charged him and Director of National Intelligence James Clapper with considering reforms to government surveillance programs. Holder last appeared before the committee in March 2013, and the upcoming hearing will cover a range of issues.