(CNN) -- The imam has stepped into the fray before.

Once, it was to address a French ban on girls wearing face-covering veils in public schools, other times, a Swiss prohibition on minarets and the burning of churches in Malaysia.

In those instances, he reminded his fellow Muslims of Islam's teachings: The burqa is a matter of style that only some Muslim countries embrace; the minarets are a matter of architecture, not religion; and firebombing churches did not sit well with Allah.

"If Muslims curse the Christians," he wrote for a Malaysian newspaper in January, "then the Christians will curse the Muslims. And people will curse Allah, and Allah will hold us responsible for that."

Such temperance is evident in many of the writings of Imam Feisal Abdul Rauf, the man behind plans to erect an Islamic center and mosque a few blocks from New York's ground zero. But in July, he turned his attention to critics of his plan -- in an effort to explain that the 9/11 conspirators no more represent Islam than abortion clinic bomber Eric Rudolph represents Christianity.

In a piece for The Washington Post's "On Faith" blog, which hosts a panel of intellectuals, clergy and journalists representing various religions, Rauf reminded readers that many Muslims -- even a Queens-reared ambulance driver -- died in the attacks on the World Trade Center.

"Religion did not distinguish the victims," he wrote. "Whether Protestant, Christian, Jewish, Muslim, Hindu, Sikh or Buddhist, or from any and no professed faith, the roll call of people who died on that terrible day reflected the diverse fabric of New York. They all died together."

Those who know Rauf describe him as a thoughtful man, a bridge builder who seeks to unite all faiths but who won't parse words when he sees religion used for nefarious ends.

His diplomacy has landed him in trouble more than once.

He has chided the U.S. for killing civilians in Baghdad. He said in 2005 that the U.S. had more Muslim blood on its hands than "al Qaeda has on its hands of innocent non-Muslims." He has refused to accept Western governments' designation of Hamas as a terrorist group. Shortly after the 9/11 attacks, he told Ed Bradley of "60 Minutes" that "United States policies were an accessory to the crime."

These are a few of the sound bites in his critics' quiver, but his supporters say the remarks are taken woefully out of context.

His colleagues describe him as scholarly, quiet, visionary, serious, tireless, tolerant, open, not frivolous, gentle and bearing a "certain air of dignity."

"I really, at times, am appalled, disappointed, staggered by the lack of knowledge about Islam in America today," said Akbar Ahmed, a professor at American University in Washington and author of "Journey into America: The Challenge of Islam."

Rauf, said Ahmed, is a Sufi. If you break Islam into three categories, you would have traditionalists, modernists and mystics. Rauf falls into the last category.

Ahmed explained the divisions. Traditionalists believe things should be exactly like they were in the seventh century, that Muslims should behave, dress and eat like the Prophet. Modernists believe Islam can be balanced with the contemporary world, through politics, education, military or policy matters. Mystics, who include Sufis, are often the most tolerant Muslims, seeking harmony as well as interfaith and intercultural discourse.

Read more about what Sufism entails

Traditionalists would support Rauf's quest to construct a mosque, but would be suspicious of the Islamic center's planned athletic rooms and swimming pool, Ahmed said. They also would frown on efforts to include Jews and Christians in dialogue.

"Converting Rauf's objective [in New York] into something like a triumphalist monument to mock 9/11 marks a profound lack of knowledge, both of Rauf and of what is happening today," Ahmed said.

However, Ahmed himself disagrees with plans for the mosque because 9/11 is central to the American identity, which should be considered in any plans to construct something near the site where almost 3,000 people were killed.

Watch Rauf defend his plans for an Islamic center

"His theology is fine. The anthropology is shaky," he said. "He's put himself and the Muslim community and supporters in a very difficult situation."

If Rauf backs down, he appears weak; if he goes forward, he appears insensitive, Ahmed said, suggesting "the Islamic way out of this impasse" would be to appeal "to the charity and compassion that lie at the heart of God."

Rather than a brick-and-mortar testament, God would prefer that his greatest creation, humans, be happy and safe, Ahmed said. It would benefit interfaith relations more if Rauf took his check for the mosque and traveled as an American imam to Pakistan, where millions suffer from recent flooding.

"Here is something far dearer to God," Ahmed said. "Let him show Muslim charity."

Eboo Patel, the Muslim founder of the Interfaith Youth Core and a member of President Obama's advisory council for the Office of Faith-based and Neighborhood Partnerships, said he believes Rauf should build the mosque wherever U.S. law allows. The First Amendment demands it.

Rauf is a major player in the interfaith world and has spoken to groups that include Christians, Jews, Hindus, Muslims and humanists, said Patel, who has seen him speak "a dozen times from Louisville to London."

Read about Patel praying with Rauf

Whether speaking in a church, mosque or synagogue, the imam begins every speech with, "My dear brothers and sisters," Patel said. Rauf has "a deep commitment" to the tenets of Islam that call for mercy, compassion and service, he added.

He attributes the negativity surrounding Islam, the mosque and Rauf to a lack of understanding. It would be no different if people lacked knowledge about Judaism or Catholicism, he said.

"Too often that vacuum of knowledge is filled with negative information," he said. "We can't paint an entire religion with a brush of ugliness and violence."

One example of negative information, Ahmed said, involves claims Rauf is attempting to foist sharia, or Islamic law, on the United States.

Ahmed, who served as a high commissioner of Pakistan to the United Kingdom, said it's confusing how civil and criminal law, tribal law, state law and sharia can run parallel in Muslim countries, but sharia has the most limited application.

"Sharia is jurisprudence, simple as that," he said.

While many Americans associate sharia with stoning women for adultery or lashing Muslims for drinking beer, Ahmed said these are Taliban-style corruptions.

In the seventh century, Ahmed said, when tribal allegiances were paramount, theft was a particularly egregious crime. Thus, sharia dictated thieves would have their hands chopped off, but it wasn't always black-and-white.

There were "safety mechanisms," Ahmed explained, and mercy was granted for a variety of reasons. If the thief had starving children or if there was a famine, among other caveats, the punishment could not be meted out.

The sharia the West hears about in Saudi Arabia or Taliban-controlled Afghanistan has been stripped of such safety mechanisms, Ahmed said. He called it a "distorted, violent interpretation of Islam," one to which Rauf would not adhere.

Patel had particularly harsh words for those who say Rauf wants to bring a brutal brand of Islam to America or that he is attempting to denigrate those who died on September 11, 2001.

Watch the mosque's money trail

"Not only is that laughable, but it's bigoted. It's patently false, and intended as a smear," he said.

Rauf has spent his life building bridges, and it's "shocking," Patel said, that his opponents would ignore his vast body of work to home in on a few inflammatory comments.

According to the American Society for Muslim Advancement, which Rauf founded in 1997, the 62-year-old imam has headed the Masjid al-Farah mosque -- about 12 blocks from ground zero -- since 1990.

Born into an Egyptian family in Kuwait, he studied in England and Malaysia before earning a bachelor's in physics from Columbia University after arriving in America in the 1960s. He later received his master's in plasma physics from Stevens Institute of Technology. He speaks English, Arabic and Malay.

His credentials are extensive, as are his writings, which include several books, "Islam: A Sacred law," "Quran for Children" and "What's Right With Islam is What's Right With America" among them.

Critics have portrayed his father as a radical, saying he was a contemporary of Muslim Brotherhood founder Hassan al-Banna.

The elder Rauf spearheaded the construction of a mosque in Manhattan and headed the Islamic Center of New York before taking over the Islamic Center in Washington.

It was at the Washington center's office complex that Rauf's father and nine others, including a reverend, were taken hostage by Hanafi Muslim gunmen in 1977, according to The Washington Post.

After the ordeal ended and Rauf's father heard gunmen had also taken hostages at Jewish and Christian houses of worship, he invited Jews to the Islamic Center, saying, "Now we are one," according to The Post's March 1977 article detailing the siege.

Watch supporters, opponents of the mosque face off

Rauf has carried on the family tradition of embracing other faiths. He teaches Islam at the Center for Religious Inquiry at St. Bartholomew's Church and at the Chautauqua Institution, which promotes itself as a place where "artists and scholars, young and old, left and right, Christian, Jew, Muslim, Buddhist, and atheist can explore, exchange, and grow."

Most recently, the U.S. State Department has tapped him to travel to Muslim countries and expand the aims of his Cordoba Initiative, which strives to build bridges between Muslims and the West.

A news release from the U.S. Embassy in Abu Dhabi last month reported Rauf spoke to a group of businesspeople and discussed the diversity of American Muslims.

"He answered questions from participants on topics including how Muslims, and especially Muslim women, can become involved in interfaith dialogue and help counter negative perceptions of Islam," the news release stated.