Back when Mike Schur was writing for Saturday Night Live–a gig that ran from 1997-2004–he says he became aware of some issues with Kevin Spacey’s behavior when the actor hosted the show. “If you think everybody knew about Louis (C.K.), everybody knew about Kevin Spacey,” Schur said, speaking on a panel with Damon Lindelof at the Vulture Festival in Los Angeles on Saturday. “I remember very distinctly being like, he’s hitting on the pages and he’s hitting on the young men in the talent department. You talk about open secret, it’s the most open secret that’s ever existed. I didn’t know maybe the extent to which the behavior was predatory.” However, Schur added everyone was aware of Spacey’s issues. “No one didn’t know,” he said.

The subject had come up in the panel discussion because Schur had taken to Twitter recently to apologize for having had Louis C.K. on his show Parks and Recreation, and not speaking up when he was already somewhat aware of the sexual harassment rumors surrounding the comedian.

“I honestly don’t remember when I first heard the rumors, but I know for a fact I had heard them before the last time he came on the show,” he said. “So in my own reckoning of my own behavior and my own past, I thought, well I didn’t say anything, I’m complicit just like anyone else is, so I felt like that sucked and I wanted to say that I was sorry, whatever that amounts to, which isn’t much.”

The big problem is, Schur said, “We just don’t talk about this stuff. Women and men to whom the behavior is being done, who it’s affecting, are scared, they don’t know who to talk to. In certain situations, in certain scenarios, there are very few other women who are even around. Even if they wanted to say something they wouldn’t know where to go. In many cases even when they did say something, nothing happened. The conversation just comes to a halt. It either never happens because the person is too afraid to say anything, or it just reaches a terminus point and it stops because everyone’s attitude is, it would be better if we didn’t talk about this, and that, going forward, is obviously something that has to change.”

For his part, Lindelof agreed everyone needs to take an honest look at themselves and their complicity. He said he particularly regretted teasing Justin Theroux during his time on Lindelof’s show The Leftovers.

“The real hard deep dive that we all have to do,” he said, “is that I think a lot of men, particularly white men in positions of power in our business, look at themselves as exceptions to the rule. They think, ‘I’m one of the good ones, I’m well-intentioned, I’m liberal. I’m not a part of the problem, I’m a part of the solution.’ But it starts with a deep dive on yourself and your past behavior.”

“For example on The Leftovers, it kind of became a running joke that Justin Theroux wore these sweatpants in the pilot, and you kind of see his genitalia bouncing around. This is something people would ask him about. He’d go on talk shows and Conan would ask him about it, or Kimmel, and he would laugh about it, and I would say, ‘Justin Theroux has a great sense of humor about this,’ so I’m going to start talking about it in the press, or I’m going to go on Instagram and say, ‘If you give us an Emmy nominations, Justin is going to take his pants off.’ And I’m not thinking in terms of, I’m actually harassing him right now, because he’s a guy, and he’s in on the joke, but you know, I never asked Justin, ‘Is this cool?’ until it had already happened, at which point I was extremely apologetic.”

Lindelof added that it’s time for everyone to take an honest look at themselves. “There’s some self-censoring that’s required,” he said.