Washington (CNN) -- As Congress debates the repeal of the Pentagon's "don't ask, don't tell" policy, military chaplains are doing their own soul-searching.

About 3,000 chaplains currently serve in the military, endorsed by a multitude of faiths, including Christian, evangelical Protestant, Jewish and Muslim denominations. It's a unique culture where chaplains of various beliefs serve alongside one another counseling and caring for an equally diverse congregation of armed service members.

"Some of the most intense and sharpest divergence of views about "don't ask, don't tell" exists among the chaplains," states the Pentagon report, released last week, on the potential impact of repealing the policy. The report concludes that allowing openly gay or lesbian troops to serve in the military would have little lasting impact on the U.S. armed forces.

Among the issues raised by chaplains, according to the report, is whether a change in policy would hinder ministers' religious expression, particularly for those faiths that consider homosexuality immoral.

"Chaplains who aren't able to proclaim what they believe is true about this issue ... means that the soldier then, the airman, the sailor, the guardian, the Marine aren't able to get the full opportunity to hear religious faiths," retired Army Chaplain Brigadier Gen. Douglas Lee tells CNN.

A chaplain for more than 30 years, Lee now serves as executive director of a Presbyterian group that endorses military chaplains from six denominations. A clergy member must be endorsed by a religious group or denomination in order to serve as a chaplain.

Lee and dozens of other former military chaplains signed a letter earlier this year to President Barack Obama and Defense Secretary Robert Gates urging them not to repeal "don't ask, don't tell."

"I fear and many others fear that down the road, knowing the other agenda items that are on the plate of those promoting a homosexual lifestyle, (there) would be a concern that chaplains would be restricted from proclaiming their faith tenets," says Lee.

Chaplains provide counseling, guidance and perform religious rites, such as baptisms and marriage, for service members.

Another retired chaplain who served for 20 years says he counseled gay and lesbian service members who confided their struggle over hiding their sexual orientation.

"For me it was very disheartening," retired Air Force Chaplain Col. Jerry Rhyne tells CNN. "I tried to bring them hope and encouragement to live their life to the fullest and to help them deal with their issues."

Rhyne, who spoke to the Pentagon group that issued the report on "don't ask, don't tell," supports the repeal, arguing it's a matter of justice.

"The scriptures and Gospels tell us we need to love our neighbors as ourselves. If you provide ministry on that basis, you can't reject somebody on their lifestyle, but you have to love them as a human being. That's a key element of providing chaplain services to military personnel wherever they are."

Rhyne says if a chaplain is unable to provide religious services because of a conflict of beliefs, they are required to find another chaplain for the service member.

"The issue should not be upon the chaplain's comfortability but it ought to be on how we provide pastoral care to the service personnel," says Rhyne.

Rhyne agrees with Gates that safeguards are in place to enable chaplains to maintain their religious beliefs and still serve a diverse community.

In a briefing last Tuesday, Gates said, "There is an obligation to care for all. But it also is clear that the chaplains are not going to be asked to teach something they don't believe in."

Despite the divergent views, the Pentagon report found that repealing "don't ask, don't tell" would have little impact on the chaplain corps. Only three out of about 145 chaplains who participated in the study suggested they would quit or leave if the law were changed.

The chaplains CNN spoke to seem to suggest the same.

"Chaplains will always help people, always have ... always will, no matter who comes their way," says Lee.

CNN's Kate Bolduan contributed to this report.