Those hopes of a Stephen Colbert/Joel McHale late-night combo? Dashed by the man himself. McHale was asked on Friday night’s Conan about if he was interested in hosting a late-night talk show. “F— no!” McHale replied. “Who would do that? That is so much work, right? I’d be worried I’d die under an avalanche of used suits.”

So much for McHale joining the late-night big leagues to replace Craig Ferguson, who’s exiting CBS’ Late Late Show next year. A picture of McHale with CBS chief Les Moonves and his wife Julie Chen posted on Twitter on the day of the broadcaster’s upfront presentation to advertisers had some people buzzing.

“Listen, that was a f— picture,” McHale said on the TBS show. “The entertainment business assumed, well, that’s a contract … take a picture with Hulk Hogan, [they’d say], ‘Well, he must be buying a spray tan company.'”

McHale added, “I’m funny about once a week,” referring to the schedule of his E! talk-show The Soup. “I’m on the E! network, where can you go higher from there?”