In the aftermath of the Donald Sterling scandal, the dirty little secret within the NBA had played itself out in a most pronounced and predictable way: For all the public declarations of support on commissioner Adam Silver's banishment of the then Los Angeles Clippers loathsome owner, there were untold peers privately peppering Silver with misgivings on this perilous new NBA world.

"Adam had far less support on Sterling than anyone knows," a league source who speaks frequently with Silver told Yahoo Sports.

All around the league, owners started to take inventory on loose memos, audio and video remnants of speaking engagements and staff meetings. From race to gay rights to fears of camera phones getting turned on them half-cocked in bars well past midnight, there were assuredly more than a few owners dispatching high-level cleaning crews to try and retrieve and expunge past indiscretions.

Dallas owner Mark Cuban had come out publicly with a fear that the rest of his peers only shared privately: Are we all headed down a slippery slope with these Sterling tapes?

View photos Former Clippers owner Donald Sterling was forced out of the league for his racist comments. (AP) More

As owners issued statements decrying the racist rants of Sterling, they feared a thousand scenarios that could unfasten them of ownership, including how it's happened to Atlanta Hawks majority owner Bruce Levenson.

Across the past several days, several high-ranking NBA officials, including owners, flew to New York to meet with Silver and discuss how the NBA would proceed on the contents of Levenson's 2012 email, sources told Yahoo Sports.

So few owners and Board of Governors members had been included in the conversation, so secretive had been Silver and league executives on the identity of Levenson, that only tiny parcels of information escaped a fortified inner circle. Through the league's back-channel gossip circles, this was known: An NBA owner is in deep trouble, and as one high-ranking official told Yahoo Sports on Friday, "I'm told it's Sterling-esque in nature."

Fear spread rapidly, because without knowledge of Levenson's identity, more than one owner wondered: "Do they have something on me?" Other high-ranking officials in organizations wondered, "Do you know if it's my guy?" Until the NBA issued a statement – deftly buried within hours of the NFL season's start on Sunday – the league was littered with guilty consciences bracing for the worst.

Ownership trepidations go far past race missteps and into possible personal failings and transgressions. What else could be cause to lose a team?

Everyone accepts this fact: Sterling's banishment was a lifetime achievement award on racism, discrimination and a downright despicable life. For legal reasons, the NBA could never make it about anything but that deranged old man's words to his girlfriend, but everyone knew the truth. The tapes had come public, and Silver had a chance to make him go away once and for all. From a business perspective, the NBA would've struggled to function without that action – from player revolts, sponsorship withdrawals and legitimate public disgust – Silver had to make his stand.

The Levenson case was a far more complex issue, and Silver was spared a far more divisive fight with owners had Levenson been unwilling to simply bow out and sell his stake in the Hawks. He had no public history of racial issues, and his 2012 email to general manager Danny Ferry clumsily tried to make sense of a legitimate business issue: How could the Hawks make game-night inclusive for a diverse Atlanta population? In some segments of the email, Levenson decries a racist attitude.

Story continues