Republican presidential candidate Ben Carson is going after an influential U.S. Islamic group, arguing it violated the federal tax code after its executive director recently called on Carson to withdraw from the race following his controversial remarks about Muslims.

The retired neurosurgeon started a petition on Thursday asking supporters to encourage the Internal Revenue Service to take away the Council on Islamic-American Relations’ tax exempt status.

RELATED: Carson jokes about taunting cops ‘before they would shoot you’

By demanding he drop out of the race, Carson said the organization had “brazenly violated IRS rules prohibiting tax-exempt nonprofits like CAIR to intervene in a political campaign on behalf of – or in opposition to – a candidate.”

“Under the Obama administration, the IRS has systematically targeted conservative nonprofit groups for politically motivated audits and harassment,” he added. “The agency should now properly do its job and punish the real violators of America’s laws and regulations.”

Ibrahim Hooper, communications director for CAIR, pushed back against Carson’s claim that the country’s largest Muslim civil rights and advocacy organization did anything wrong.

“We find it interesting that Dr. Carson seeks to use a federal government agency to silence his critics and wonder if that tactic would be used to suppress First Amendment freedoms should he become president,” Hooper told MSNBC.

“CAIR is not in violation of any IRS regulation in that we did not ‘participate in’ or ‘intervene in’ any political campaign. We, as mandated by our mission as a civil rights organization, merely expressed the opinion of our community that a candidate whose views violate Article VI of the Constitution is unfit for public office,” Hooper added.

RELATED: Ben Carson raises $20 million in third quarter

The war of words between CAIR and Carson began late last month after the GOPer said on NBC’s “Meet the Press” that he “would not advocate that we put a Muslim in charge of this nation.” A day later, he seemingly tried to walk back his remarks on Fox News, saying he would back a Muslim for president, but only if that person was “willing to reject those tenets and accept the way of life that we have and clearly will swear to place our Constitution above their religion.”

Still, Nihad Awad, national executive director of CAIR, called on Carson to withdraw his name from the presidential race, arguing he is “unfit to lead.”

According to Real Clear Politics’ national average of polling data surrounding the Republican presidential nomination, Carson is currently in second place with 16.3% support, several points behind real estate mogul Donald Trump, who is leading the field with 23.3%.