RICHMOND, Va. -- Protesters gathered outside the Executive Mansion by the dozens Monday to urge Virginia Governor Ralph Northam to step down.

Carrying signs and chanting, the protesters said they believed Northam was no longer fit to lead the people of Virginia.

"We have a leader that’s not listening to us," protester Tracey Cortder said. "We have a leader that thinks their personal ambitions is more important than the good of the state."

Monday's protest came after a weekend which saw Northam initially saying Friday night that he was in the photo that appeared on his page of his 1984 medical school yearbook which shows one person dressed in blackface and another in the KKK's signature white hood and robes. On Saturday, Northam was insistent that he is not one of the two people pictured.

"I think back to 1984. I was five. At five years old I knew that was wrong," another protestor chimed in. "He was a grown man in medical school, so yeah. That’s not someone I want representing us."

Many who spoke at Monday's protest mentioned they weren't condemning Northam as a man, but holding him accountable for his actions.

"One of my core values is redemption, and I believe redemption should happen outside of office," protester Tracey Corder said.

"There's some learning and some growing that he should be doing when he`s not legislating for the state."

Others called the scandal a tipping point in a larger issue of racism and discrimination throughout Virginia.

"He has opened up the flood gates for us to take action," a protest organizer said.

Northam, a Democratic, said at an all-staff meeting Monday that he needed more time to decide his path forward.

Meanwhile, Lt. Governor Justin Fairfax is facing his own scandal; allegations of sexual assault.

He says he hasn`t spoken with Northam in a couple days, and doesn't know why he has chosen not to resign.

"I have no idea what the Governor`s thinking is on that front," Fairfax said. "You`d have to ask him."

Speaker Kirk Cox says removing Northam from office is unlikely, and once again called on him to step down.

"I would say that this is just heartbreaking," Cox said. "I just don`t want to see him going down this way."

This is a developing story.