The track record of the U.S. Secret Service in protecting President Barack Obama has come under intense scrutiny in recent months due to several major security lapses, the most recent of which prompted the resignation in October of Secret Service Director Julia Pierson.

A recently released Department of Homeland Security report about that incident determined that a number of “performance, organizational, technical and other” factors contributed to the security breach.

What is not mentioned in that DHS report is another long-running issue within the Secret Service—one which, according to several experts, creates the conditions for a breakdown in agency morale that could ultimately compromise the security of President Obama. That threat is embodied in a long-running lawsuit filed by a group of African-American Secret Service agents who allege the agency’s culture is replete with racism.

In September, a man with a knife scaled a fence and ran into the White House, making it as far as the East Room—it was this incident that prompted the report by DHS, which oversees the Secret Service. That same month, however, in Atlanta, the media reported that an armed security contractor with an arrest record was somehow allowed to ride in the same elevator as President Obama.

Those incidents echo an even more serious security breach that played out in 2011, when a gunman with a semiautomatic rifle managed to fire some seven bullets into the White House while one of the president’s daughters was home and the other expected to return that same evening—yet it allegedly took the Secret Service four days to determine the shots had been unleashed on the president’s home.

“If the black Secret Service agents’ legal claims related to racism in the agency are true, then there is a threat to the president’s security because he is a black man,” says Matthew Fogg, a retired chief deputy U.S. Marshal who in 1998 won a multimillion-dollar jury verdict in a racial discrimination lawsuit filed against his agency. “If they are treating black people differently, then how can that not affect the president?”

Fogg also is party to one of two pending class-action discrimination cases filed by black federal agents against the U.S. Marshals Service. The mere fact that the most qualified agents are not getting promoted within an agency due to racial factors—as is alleged in both the the Secret Service and U.S. Marshals litigation—indicates, Fogg says, that race is an issue in the quality of protection being provided to individuals, including the president.

Attorney Ronald Tonkin, a former federal prosecutor who now represents federal agents in whistleblower and employment-discrimination cases, says regardless of which side prevails in the Secret Service litigation, both the accused and accusers “are affected” by the resulting tension and polarization.

“These agents are all professionals, but the question is whether their professionalism is dissipated by a perception that they are being treated badly,” said Tonkin, who also serves as associate counsel for the National Association of Federal Agents. “It does, in my experience, have an effect on agency morale, and that affects performance at some level.”

The black Secret Service agents’ litigation, certified as a class-action lawsuit last year, has been pending in federal court since 2000 due to numerous procedural delays. It alleges that black agents, as a group, have been systematically discriminated against in hiring, assignments, transfer, awards, promotions and discipline.

“The Secret Service has failed to protect its African-American special agents from racial discrimination in virtually every aspect of their employment,” the black agents’ pleadings in the case allege. “Discrimination against African-American agents in the Secret Service has become part of the fabric of the agency and has spanned several decades.”

A complaint filed by the 10 black agents who are named plaintiffs in the class-action suit points to an event held in southeastern Tennessee called the “Good Ol’ Boys Roundup,” which was made public in the mid-1990s via media reports but had been staged annually since 1980.

“The racist conduct that occurred at the Roundup included the posting of racist signs like ‘Nigger checkpoint,’ a simulated lynching of a black man from a tree, and a host of racist skits and songs,” the complaint alleges. “Officials at the Secret Service knew about this event, and at least 30 Secret Service agents were documented attending the event. … Many of the white special agents who attended the Roundup were promoted to high-ranking positions in the Secret Service, including three agents who became SAICs [special agents in charge] of field offices … and two agents who were promoted to the Senior Executive Service level.”

Reginald Moore, special agent in charge of the Secret Service’s Atlanta field office, and a party to the black agents’ class-action lawsuit, declined to comment on the case or any potential threat to the president due to the alleged racism within the agency. Likewise, Angela Burns-Ramirez, a former Secret Service special agent who has an individual racial-discrimination complaint pending in federal court against the agency, declined to comment on the advice of her attorney. A primary allegation in her lawsuit, which mirrors claims raised in the Secret Service class-action litigation, is that she “has been treated differently and subjected to different terms and conditions of her employment due to her race (African American).

“Despite being established in 1865, the first African-American female was not promoted to a GS-14 [supervisory] position until 2001,” Burns-Ramirez’ court pleadings state. “Only five African-American females hold a rank higher than GS-14 within the Secret Service.”

Brian Leary, a spokesman for the Secret Service, declined to discuss the black agents’ class-action lawsuit or the impact their allegations might be having on the agency. “We aren’t commenting due to the ongoing nature of the lawsuit,” he said.

Professor Richard Delgado, the John J. Sparkman Chair of Law at the University of Alabama School of Law, warns that it would be a mistake to discount the effect that racism can have on an institution and its ability to function effectively. Delgado is a leading scholar in the field of Critical Race Theory, which focuses on the ways in which racism is embedded historically in the nation’s laws and legal system.

When racist activity is claimed, “it creates a terrible environment for both sides of the color line,” Delgado explained.

“For the minority agents who feel discriminated against, they feel unappreciated and develop a defensive attitude at best,” Delgado added. “Many of their white colleagues see the minority agents as troublemakers scheming to get ahead, talking to lawyers.

“Both sides are certain they are right, and it’s a recipe for morale problems or worse. It can lead to inattention on the job or even small acts of sabotage designed to make someone look bad.”

Delgado said he would not want to be “a federal official with an agency in that state charged with looking after my back.”

The solution in such a case, he said, isn’t simply to ask people to set aside their animosities, because “that doesn’t work.”

“The best course is to arrange a lot of contact early in life, so that blacks, whites and Latinos get to know each other and see we’re all humans with a range of behaviors and character, some good and some bad,” Delgado stressed.

“And these agencies should be screening people early, before they are hired, to determine if they, regardless of color, have the right attitudes and ability to work well in diverse groups.”