Thomas Ravenel is alleging that ex-girlfriend Kathryn Dennis has sometimes put the care of their two young children in the hands of her boyfriend, Hunter Price.

As Southern Charm fans recall, she has been fighting for sole custody of daughter Kensie, 5, and son Saint, 3. Dennis, 26, filed for a modification of their current 50/50 agreement back in October after Ravenel, 56, was arrested and charged with assault and battery. (He has denied any wrongdoing through his attorney.)

Get push notifications with news, features and more.

She requested that Ravenel’s on-again, off-again girlfriend Ashley Jacobs be prohibited from spending time with the kids, too. But in court documents filed on April 29 and obtained by The Blast, Ravenel suggested that her request is unfair given Price’s contact with their children.

Reps for Dennis and Ravenel did not immediately respond to PEOPLE’s request for comment.

Ravenel claims that Price — a Nashville-based singer-songwriter and America’s Got Talent alum — has “moved in” with Dennis in Charleston, South Carolina, and stayed at her house overnight while the children are in her care. He also alleges that Price sometimes takes care of the children while Dennis is not around.

Dennis and Price, 26, made their relationship public in January, though it is not clear when they started dating. Ravenel claimed in his filing that Dennis violated their order by permitting him around the children before they’d been seeing each other for four months.

Ravenel alleges that Dennis’ attempt to limit Jacobs’ time with the children is “a blatant attempt to control [his] personal life and place restrictions on who he dates.” Ravenel claims that Dennis “has a history of jealousy and control issues and has never approved of any female [he] has dated” and has allegedly disparaged him to their children, including telling daughter Kensie that he “tried to hurt and kill” Dennis.

The controversial politician is asking the court to suspend Dennis’ visitation with the kids until Price moves out and refuses to stay there overnight. Ravenel also wants Dennis’ visitation cut from every other week to every other weekend.

As for child support, though Dennis had requested that Ravenel start paying her support (she currently pays him a monthly stipend), he claims she makes “six figures” and can afford their expenses.

If Ravenel’s request is granted, Dennis would be subject to continued random drug tests as well, and would not be able to film with their children on Southern Charm or post images of them on social media.

Image zoom Thomas Ravenel Rodolfo Martinez/Bravo/NBCU Photo Bank via Getty

This is one of three legal battles Ravenel is currently involved in.

In November 2018, he sued Bravo and Haymaker Media, Southern Charm‘s production company, attempting to stop them from broadcasting unaired footage of his children on the show. There has not been any public update on the case.

Additionally, he claims Bravo and Haymaker “encouraged” Dennis to trying to get primary custody to create drama for the show. She famously lost full custody of her children in 2016 after failing a drug test for marijuana, but has slowly earned small percentages of it back by going to rehab and staying clean.

A judge has ordered Ravenel to go to trial on the criminal sexual assault charges, though he has not yet entered a plea.