(CNN) Don't expect to hear Matt Damon weighing in on Hollywood's sexual harassment reckoning any time soon.

Just weeks after the "Downsizing" actor found himself in hot water over his comments about what's come to be known as the #MeToo movement, Damon says he's going to recuse himself from the conversation for a while.

"A lot of those women are my dear friends and I love them and respect them and support what they're doing and want to be a part of that change and want to go along for the ride, but I should get in the back seat and close my mouth for a while," he told Kathie Lee Gifford on the "Today" show on Tuesday.

Damon was slammed back in December after he told ABC News' Peter Travers in an interview that while women feeling empowered to share their stories of sexual harassment and mistreatment is "totally necessary," he hoped for more attention to be paid to the "spectrum of behavior."

"There's a difference between, you know, patting someone on the butt and rape or child molestation, right? he said. "Both of those behaviors need to be confronted and eradicated without question, but they shouldn't be conflated, right?"

Read More