As calls mounted for his resignation after three failed attempts to explain away a racist photo on his yearbook page, Virginia Gov. Ralph Northam reportedly told a Cabinet meeting Monday he’d be seen as a “racist for life” if he were to step down. The governor, a Democrat, doubled down even as the African American Lt. Gov next in line for the office, fellow Democrat Justin Fairfax, was responding to claims about his own past from the same right-wing oppo shop that had first reported on the governor's yearbook.

While no Cabinet members directly told Northam to resign, those present at Monday's meeting told him that staying in his current position wouldn’t work, according to a source cited by CNN.

"We are struggling here, there is no better way to put it. We all love the governor," a source in the room told CNN. "We don't think he has a racist bone in his body—but he can't seem to explain this, and it is making it very difficult for us to stick by him."

Northam has already suffered a huge political blow, with his approval rating plunging 41 percentage points in the two days after the yearbook page was first featured in Big League Politics, according to Morning Consult. Northam’s disapproval rating shot up to 48 percent overall, gaining 22 points compared to numbers collected in January polling.

His apparent political toxicity caused College of William & Mary's to cancel his scheduled speaking events at the university’s Charter Day and inauguration ceremony on Friday.

“...Under the circumstances, it has become clear that the Governor’s presence would fundamentally disrupt the sense of campus unity we aspire to and hope for with this event,” President Katherine Rowe said in a statement Monday, according to Politico. Rowe also reportedly said she was “appalled” by the image, which she claimed has “no place in civil society.”

Complicating matters, however, Fairfax, who would become governor if Northam were to resign, is now under scrutiny after a sexual assault allegation against him from 2004 was outlined in Big League Politics, the same outlet that published Northam's yearbook photos.

Fairfax has denied the allegations in a statement and described a "consensual" encounter, saying the allegation of assault was “false” and claiming he “never assaulted anyone—ever—in any way, shape or form.” He also said The Washington Post had previously interviewed the same accuser and found “significant red flags and inconsistencies within the allegations,” a claim which the Post promptly denied. In response to Fairfax's statement, the Post refuted his claim that "red flags" had been found in the allegations and reported that the same accuser had approached the newspaper in late 2017 but the paper had ultimately not published a story because it could not independently corroborate the accuser's account.

The accuser alleged that Fairfax forced her to perform oral sex after they met at the Democratic National Convention in 2004, and the incident left her “crying and shaken.”

The newspaper's executive editor, Marty Baron, said in a statement of his own Monday that the paper had finally published a story on the allegations because it had an “obligation to publish” the claims to clarify Fairfax's statements and in light of the fact that Fairfax "may well rise to the position of governor."