Washington (CNN) National security adviser John Bolton's new chief of staff has drawn scrutiny for his past work with the Center for Security Policy, a conservative think tank that has been accused of promoting anti-Muslim messages.

Fred Fleitz, who joined the National Security Council this week, was an outspoken opponent of the Iran nuclear deal when he worked as the senior vice president for policy at the think tank led by Frank Gaffney, whose views on Islam and theories about the Muslim Brotherhood's influence have sparked controversy.

Fleitz was one of more than a dozen authors of a 2015 report from the Center for Security Policy that called for, among other things, using "shariah-adherent advocacy and practices as legal premises for deportation and stripping of American citizenship."

The report also argued academic institutions that accept donations from "shariah-adherent individuals or governments" should be "discouraged" from promoting things like "interfaith dialogue" or "Muslim-Christian understanding," because that effort "amounts to promoting shariah or civilization jihadist agendas."

In an interview with Breitbart News last year , Fleitz also blamed "the failure to assimilate British Muslims" in the UK for a terror attack in London.

Fleitz did not respond to a CNN request for comment.

'Middle of the road conservative'

Stephen Elliott, a former George W. Bush administration official who worked with Fleitz at the State Department, calls Fleitz a "middle of the road conservative" who has tough positions on Iran, North Korea and terrorism.

"He is not out of the mainstream. He was tough on Iran, but so was the Bush administration," Elliott said. "Taking hardline positions on the issues of the day doesn't make him a neo-nazi or a wing nut as people have called him, which is outrageous."

Fleitz's policy views at the State Department and since then have been focused on "radical Islam, not all Islam," Elliott said.

"He has written four books, countless articles and had many TV appearances. Maybe you can argue he has been too tough, but he makes a clear distinction between radical Islam and mainstream Islam," the former Bush administration official said. "I know it is easier to pick out something that sounds ominous and scary. But it is unfair to tar him with this report when there is no lack of information on what he has said and what he has written."

Criticism and Frank Gaffney ties

Fleitz's ties to Gaffney and the Center for Security Policy have raised concerns among a number of advocacy groups, including the Southern Poverty Law Center and the Anti-Defamation League

The Southern Poverty Law Center is an Alabama-based nonprofit activist group that tracks civil rights and hate crimes and defines a hate group as an organization with "beliefs or practices that attack or malign an entire class of people, typically for their immutable characteristics."

The group slammed Bolton's decision to hire Fleitz this week.

"(T)he CSP is known for propagating wild conspiracy theories that have no basis in fact but do stoke a climate of fear and violence against American Muslims," the Southern Poverty Law Center said in a statement on Wednesday. "Fleitz will now be in a position to do serious harm to our national security as he pushes an extreme anti-Muslim agenda."

A senior administration official dismissed the focus on Fleitz's past publications as a "baseless" attack and blamed the Southern Poverty Law Center and other groups "for waging a vengeful smear campaign for Fleitz's past criticisms of the group, which has been known to attack prominent conservative thought leaders and label them bigots."

"The criticisms that I've seen at least are very familiar," Gaffney told CNN. "They are the kinds of things that one has come to expect from what I believe is a discredited instrument of political warfare," Gaffney added, referring to the Southern Poverty Law Center.

"I think Fred has certainly been exposed to these arguments and has been a contributor to some of the work that we've done over the years," he said.

Defenders

Fleitz's relationship with Bolton stretches back more than a decade. The former CIA analyst served as Bolton's chief of staff in the Bush administration and went on to work as a senior staffer on the House Intelligence Committee in 2006. Bolton served as Bush's ambassador to the United Nations.

Sebastian Gorka, a former deputy assistant to President Donald Trump, defended Fleitz's past writings about Islam and also blamed the Southern Poverty Law Center for attacks on Bolton's new chief of staff.

"Fred Fleitz is one of the few people since 9/11 who has been trying to speak truthfully about the global jihadi movement. He's been pushing back on the political correctness that has gotten people killed," Gorka told CNN. "I think the Southern Poverty Law Center is one of the most radical hate groups in America today."

Jim Hanson, a former executive vice president at the Center for Security Policy and current president of the Security Studies Group, denied Fleitz is Islamophobic.

"I worked with Fred Fleitz at CSP and his primary responsibility was the Iran nuclear program. He has done some of the best work exposing the nature of that threat and how the Iran deal actually made it worse," he said. "Those attempting to paint him as Islamophobic are mistaken. He is not and he rarely focused on the Islamist issues except as they related to malign activities of the Iranian regime."

CORRECTION: This story has been updated to correct the spelling of Stephen Elliott's name.