Matt Damon expanded on his recent comments about the climate of sexual misconduct allegations in Hollywood by arguing that the “preponderance” of men he’s worked with are not predators and by saying that he’d decide whether to work with people accused of inappropriate behavior on a “case-by-case basis.”

Speaking with Business Insider, the actor returned to the topic while promoting his new film Downsizing, which opens in select theaters Friday. “We’re in this watershed moment and it’s great but I think one thing that’s not being talked about is there are a whole sh–load of guys — the preponderance of men I’ve worked with — who don’t do this kind of thing and whose lives aren’t going to be affected,” Damon said. “If I have to sign a sexual harassment thing, I don’t care, I’ll sign it. I would have signed it before. I don’t do that and most of the people I know don’t do that.”

Business Insider then asked Damon whether he’d back out of a movie if someone prominent involved, such as an actor, director, or producer, had been accused of sexual misconduct. “That always went into my thinking,” Damon responded. “I mean, I wouldn’t want to work with somebody who — life’s too short for that. But the question of if somebody had allegations against them, you know, it would be a case-by-case basis. You go, ‘What’s the story here?'”

Last week, Damon incited controversy after discussing the topic with critic Peter Travers. He similarly championed the “watershed moment” of women coming forward while also cautioning against conflating “patting someone on the butt and rape or child molestation.” He particularly compared the allegations that have been made against Senator Al Franken, who recently announced he’d be resigning from the U.S. Senate, and Harvey Weinstein. “When you see Al Franken taking a picture putting his hands on that woman’s flak jacket and mugging for the camera… that is just like a terrible joke, and it’s not funny. It’s wrong, and he shouldn’t have done that,” Damon explained. “But when you talk about Harvey and what he’s accused of, there are no pictures of that. He knew he was up to no good. There’s no witnesses. There’s no pictures. There’s no braggadocio. That stuff happened secretly, because it was criminal and he knew it. So they don’t belong in the same category.”

Franken was accused of sexual misconduct by a half-dozen women after broadcaster Leeann Tweeden first came forward with a 2006 photo showing the politician grabbing for her chest while on a USO tour. In a statement, Franken apologized for his alleged behavior. “Some women have found my greetings or embraces for a hug or photo inappropriate, and I respect their feelings about that,” he said. “I’ve thought a lot in recent days about how that could happen, and recognize that I need to be much more careful and sensitive in these situations. I feel terribly that I’ve made some women feel badly and for that I am so sorry, and I want to make sure that never happens again.” Franken resigned from the Senate this month.

Weinstein, who produced Damon’s Oscar-winning Good Will Hunting, has been accused of sexual misconduct by more than 60 women, including claims of assault in many cases. In a statement provided to EW earlier this month by attorneys Blair Berk and Ben Brafman on behalf of Weinstein, the producer denied any allegations of sexual assault: “Mr. Weinstein has never at any time committed an act of sexual assault, and it is wrong and irresponsible to conflate claims of impolitic behavior or consensual sexual contact later regretted, with an untrue claim of criminal conduct. There is a wide canyon between mere allegation and truth, and we are confident that any sober calculation of the facts will prove no legal wrongdoing occurred. Nonetheless, to those offended by Mr. Weinstein’s behavior, he remains deeply apologetic.”