The disgraced politician was arrested and charged with assault and battery in the second degree on Tuesday

Thomas Ravenel will not be returning as a Southern Charm cast member next season, Bravo confirms to PEOPLE exclusively.

Get push notifications with news, features and more.

The network’s decision comes after Ravenel suggested he was quitting in August, tweeting “I’m not doing the show anymore. In the contract they have the right to fictionalize your story. They took advantage of me. I decided I’ve got too much to lose and informed them I’m not coming back.”

Police began investigating Ravenel in May after a woman reported him for sexual assault. A former nanny to his children named Dawn claimed to PEOPLE that she is that woman and alleged that, while his daughter was sleeping nearby, he “corralled” her into the master bedroom, dropped his pants, blocked the door and “proceeded to rip my clothes off.” Without going into further detail, Dawn said she eventually got away.

A source tells PEOPLE that Ravenel’s arrest is connected to her allegations.

RELATED VIDEO: Southern Charm’s Thomas Ravenel Accused of Sexual Assault, Allegedly Paid Accuser $200,000

Ravenel denied Dawn’s allegations through his lawyer, as well as a separate sexual assault accusation from a woman he met on Tinder in 2015.