Rick and Morty has a toxicity problem among its fan base — one that co-creator Dan Harmon is aware of.

Polygon spoke to Harmon at New York Comic Con about a few of his series, including the second season of HarmonQuest and the recent finale of Rick and Morty’s third season. Harmon, who just weeks prior told Entertainment Weekly his views about the various fans harassing female writers on the show, addressed the problem that exists among some of Rick and Morty’s fanbase. A popular phrase that runs amok throughout certain subreddits, that Harmon is aware of, is that “Rick and Morty is for smart people,” which has been used to call out the most recent season for flaws that viewers have found with it.

Harmon told Polygon that although he “works in a cave,” he does hear about these things from other people and it bums him out.

“I know there’s toxicity among the fan base because it’s a popular show and people obsess about it,” Harmon said. “We try to outdo each season. If it was dumber, it was an accident. Maybe I got dumber! You can’t argue with somebody who says they like this season, but not that season. It’s not the same show season to season.”

Harmon is a quiet man, but when he’s passionate about a certain topic, he’s ready to discuss it at length. The subject of toxicity that lies within a few very vocal Rick and Morty fanbases irks the creator, but he’s aware that, much like the internet, there are fewer sour apples than pleased viewers. Much like the internet, the angrier Rick and Morty fans just happen to be louder.

“I think if you’re talking to someone about whether or not something sucks it’s a good indication that they’re not watching TV right,” Harmon said. “They’re probably a little swept up and — I’m not saying they’re wrong to be this invested in a show — but I wouldn’t trust that person to tell me what’s funny because they’re kind of scowling already.”

“It’s a popular show and people obsess about it”

During our conversation, Harmon remarked that he wasn’t immune to the feelings fans had about the show. Like them, Harmon said he understood what it was like to be so invested in a series or franchise that you felt like you had to defend it.

“Look, I’m one of these guys,” Harmon said. “I grew up shaking my fist at this Star Wars movie over that Star Wars movie ... as long as I can keep making this show, hate it all you want. I’d be lying if I said I wasn’t trolling a little bit by making sure that a season they had political reasons to hate was one of the best that we did. Then again, I can’t manipulate the quality of a show season to season. But what I can do is make sure it’s a really good season.”

Harmon added that he tries to stay out of it, taking prolonged breaks from Twitter or social media and clearing his head. For the most part, he said, people are more than welcoming of the new season and that’s what matters. Harmon said he’s proud of the season they completed, reiterating over and over again during the course of our conversation that he thought it was the best one they had made.

Rick and Morty’s third season is available to watch on Adult Swim’s website for cable subscribers.