Stephen Colbert has had a lot of tough gigs in his career, but he says nothing compares to his live Election Night special for Showtime, which became more of a wake than a comedy show due to Donald Trump's surprise victory.

"That show was the hardest thing I've ever done in my entire life," Colbert said Saturday night. "The audience was sobbing openly."

Colbert made his comments while holding an election postmortem with fellow late-night host (and Daily Show alum) John Oliver, as part of a Montclair Film Festival fundraiser at the New Jersey Performing Arts Center in Newark, N.J.

In the leadup to his Showtime election night show, which Colbert was able to do because CBS had preempted The Late Show with Stephen Colbert that night for election coverage, "we had gone over every possible eventuality. We had so many guests, we had so many pretaped pieces, all based on a different eventuality," including that Hilllary Clinton won, or Clinton was the likely winner but the race was too close to call, or that Trump was possibly going to win.

"And then there was the last show, the show we did, Donald Trump is going to win and we know he's going to win. And then execs and my writers were like, 'You don't want to write something for that?' And I was like, 'No!'" said Colbert, explaining that performing jokes about Trump winning the presidency for his studio audience of 400 would be like doing standup comedy during an execution.

Of course, that's pretty much what unfolded that night. "Over my guests' shoulders, people kept putting up signs: Florida, Ohio, Iowa, Nebraska," recalled Colbert, referring to the states that Trump had won. "We only did about 20 minutes of material before we went, fuck it, it's going to be him, let's just talk for another hour. We have two and a half whole shows that you will never ever see of material that we had to kill that night."

That unaired material included a number of pretaped pieces and a parade of naked men "with high, tight butts," said Colbert. "They were going to going to come out there and painted on their asses, it said, 'I'm with her, exclamation point.'"

Colbert also addressed the recent controversy around fake news on sites like Google and Facebook, and said he and Oliver took issue with the term. "Because what we did was fake news" as correspondents on The Daily Show, Colbert said. "The fact that they call this stuff fake news upsets me, because this is lying. This isn't 'fake news.'"

The hosts also pushed back against those who claim that a Trump president must be a late-night host's dream scenario. "The next person who says, 'You must be happy on a certain level,' is going to have their eyes carved out. It's not fun," said Colbert.

Oliver agreed, noting the difficulty his show had during the election, when "you try and take take things of substance and then put some sugar on it to make it palatable. But there was so little substance this whole campaign that there was just a diabetes-inducing amount of sugar." Unable "to find a way to inject substance into sugar," Oliver said his team instead "just tried to think about framing devices, how to make something out of almost nothing."

As they considered what President Trump will be like, Colbert referred to everyone asking residents to give Trump a chance as president. "I'm all for give him a chance, but don't give him an inch. Because I remember everything he said and it's horrifying," he said.

The duo talked about more than just politics. Oliver spoke at length about the research that goes into his Last Week Tonight segments, but admitted, "The happiest I ever am is when we do something just unspeakably dumb. …a gigantic waste of HBO's resources. We built an entire dog Supreme Court!"

Oliver also talked about a horrible commercial audition he had for a deodorant commercial in England, early in his career. "They said, okay, now we need you to take your shirt off. So I took it off, and as I'm doing it, thinking I can feel me losing the job," said Oliver (he was right).

Every fall, Colbert interviews a friend like Oliver for the annual Montclair Film Festival fundraiser (past guests have included Jon Stewart, Steve Carell and JJ Abrams). The host said he hopes to speak with another former Daily Show correspondent, Samantha Bee, at next year's fundraiser.