For all the years it has been America’s most-watched network, CBS’ secret weapon was always recognized in the industry to be the stability of core management under the leadership of Leslie Moonves.

The loyalty of Team Moonves, populated with many execs who had been with him since the Lorimar Television days in the 1980s and early ‘90s, was the envy of CBS’ Big Four rivals. ABC, NBC and Fox have had revolving doors of executives in the 24 years that Moonves has steered CBS.

But a different picture of Moonves — who rose from entertainment president in 1995 to corporate CEO in 2006 — is emerging now that he has been forced out on under a cloud of scandal and a cascade of sexual assault and misconduct allegations. Company insiders, current and former employees, describe an atmosphere of fear in which executives were on often edge around Moonves. Sources say the bonding among the core team came in part from banding together to strategize on how to best manage Moonves: One former executive described him as serving, at times, as a “common enemy” who helped bring them together.

The company was known to have a familial environment, particularly during the 1998-2015 period when Nancy Tellem and Nina Tassler led the Los Angeles-based entertainment division. Tellem and Tassler declined to comment for this story.

But while everyone at CBS knew about his volatility, the details about sexual assault in the New Yorker exposes published July 27 and Sept. 9 by investigative reporter Ronan Farrow were “stomach-churning” to many of those who worked closely with Moonves. The violent behavior alleged by some of the women who spoke to Farrow was alarming and hard to reconcile with the charismatic, if quick-tempered, leader they knew. A source with knowledge of the situation said Moonves has staunchly denied the allegation by former TV executive Phyllis Golden-Gottlieb, reported in Farrow’s Sept. 9 story, that he threw her against a wall when they worked together at Lorimar Television in the mid-1980s.

Legal and PR representatives for Moonves did not respond to multiple requests for comment on this story. On Sept. 9, Moonves issued a statement asserting: “Untrue allegations from decades ago are now being made against me that are not consistent with who I am.” He also said he was “deeply saddened” to be leaving CBS and the team he assembled. “The best part of this journey has been working alongside the dedicated and talented people in this company,” he said.

Farrow’s reports have prompted soul-searching among current and former executives. There’s been a lot of communication among past and present CBS employees as the Moonves scandal went nuclear during the weekend, leading to his Sept. 9 ouster, hours after Farrow’s second report was published. Current and former employees have been ruefully comparing notes and questioning whether there were warning signs that should have raised alarms. There is also a swelling anger that the reports of Moonves’ behavior have cast a dark cloud across the whole of CBS. “It’s infuriating to think that people think that this is what the culture of CBS is,” said a senior L.A.-based executive. “It’s not. This is not going on behind every other door here. It’s just not.”

Moonves’ core team collaborated in the formidable job of running the network, and also on the best ways to handle a temperamental boss. The higher up executives climbed and the longer they worked with him, the more staffers were exposed to the darker side of his forceful personality. His eruptions sometimes got so loud and intense as to frighten the recipient, multiple sources said. “Scary screaming,” one described. His mood swings made the already high-stakes atmosphere of running a television network even more fraught. “You would just be fearful of what kind of Les you would get,” said another CBS veteran.

His toughest critics do not disparage his programming instincts or business acumen. “He has incredible instincts on shows and on deals,” said a longtime colleague of Moonves. But there was often frustration that Moonves received so much of the credit for CBS’ successes, in programming and in business. Insiders credit the company’s sterling reputation on Wall Street largely to the heavy lifting done by Joe Ianniello, the former chief financial officer and chief operating officer, who was tapped as acting president-CEO on the heels of Moonves’ forced resignation on Sept. 9.

Moonves gave executives he believed in big opportunities to advance their careers. But he would “torture” them at the same time, in the words of one CBS alumnus. Executives in programming and marketing-related jobs had to be accustomed to incessant micro-managing, along with second guessing, threats of being fired, and ultimatums about what the failure of a show or a business venture would mean for a person’s career at the company.

It was a corporate hazing that toughened the hide of every executive in Moonves’ direct orbit. It also created a kind of Stockholm syndrome environment for senior executives. Multiple sources described the process of deciding to quit as difficult and one that had to be carefully managed with Moonves, lest he feel it was a snub. He was known to be vindictive to those he felt had crossed him.

As tough as he was, Moonves could be generous with his time and rewards for those who delivered for CBS. He was a savvy mentor for many who aspired to rise in the programming department. He was at his best at times of stress and crisis. He was adept at getting everyone “rowing in the same direction” during busy periods such as preparing for the annual spring upfront presentation, or during the upheaval after the 9/11 terrorist attacks. He also responded well to executives who demonstrated a passionate belief on a particular matter, whether it be the potential of an up-and-coming actor or the merits of a script. “He would listen to you and if you’d convinced him he’d say, ‘OK but your ass is on the line if it doesn’t work,’” one CBS veteran said.

As Moonves rose through the corporate ranks in the early 2000s, his reputation as a womanizer was well-established. It was known that he had at least one consensual relationship with a subordinate female CBS executive during the period when he was separated from his first wife, Nancy Moonves. But the allegations of violence and attempts to force himself on women still shocked many of those who have worked with him for years. As disturbed as CBS veterans are about those allegations, several of those reached for this story said they still had concern for Moonves’ well-being amid the turmoil. Some also characterized the reports of his temper as overstated.

Multiple sources said Moonves assured senior CBS executives that he was not vulnerable when the first wave of sexual abuse exposes began to surface last fall with reports on Harvey Weinstein, Kevin Spacey and other now-disgraced figures.

Moonves last week admitted to trying to kiss a UCLA physician Anne Peters in 1999 during an appointment although he denied the assertion Peters made in a medical journal (without naming Moonves) that he “satisfied himself” in her presence after trying to force himself on her. “The appalling allegations about my conduct toward a female physician some 20 years ago are untrue. What is true, and what I deeply regret, is that I tried to kiss the doctor. Nothing more happened,” Moonves said in a statement to Vanity Fair, which first reported on the incident.

Within CBS, there’s growing anger that the fact that the allegations about Moonves’ actions have at least temporarily put a taint on the legacy of the entire team. “Betrayal” was a word used by multiple sources. There is also resentment that Moonves held on during the past few weeks without stepping down to avoid the embarrassment of Sunday’s hasty departure on the heels of Farrow’s expose. The hit that CBS’ stock price has taken is at least a temporary blow to executives with stock options and grants.

In recent years, Moonves’ grip on CBS’ day-to-day operations loosened slightly. It was clear to those closest to him that he was thinking about his next chapter and a graceful segue. It’s understood that about two years ago there was some discussion of Moonves succeeding Roger Goodell as NFL Commissioner as his transition out of his CBS. The move was said to have been supported by Robert Kraft, owner of the New England Patriots and a friend of Moonves, but the opening at the league never materialized.

Those who know Moonves well say the former actor loved nothing more than playing the role of Leslie Moonves, leader of CBS. He had become attentive to his own considerable legacy as a programmer in the past few years, seeking industry recognition for his highly successful run that is nearly unprecedented for an executive at a single network. He loved being compared to CBS founder William S. Paley, whose office at CBS’ New York headquarters Moonves occupied, until Sept. 9. But his legacy is now irreparably tarnished, a downfall that has stunned the industry.

As the initial shock begins to subside, some CBS insiders are expressing cautious optimism about the prospect of gaining a little more autonomy in their day-to-day work lives. And some are energized to move the company forward after the massive shakeup that included six new board members, and a new permanent CEO still to be named. “People are looking forward to getting to live or die by their own decisions,” a CBS veteran said.

Disclosure: Cynthia Littleton co-authored a book with Nina Tassler, “What I Told My Daughter: Lessons From Leaders on Raising the Next Generation of Empowered Women,” published in 2016 by Simon & Schuster, which is owned by CBS.