On Tuesday, Stephen Colbert will host “The Kennedy Center Honors” on CBS, the second time the 51-year-old funnyman has hosted the special since the network named him (to much fanfare) as David Letterman’s replacement on “The Late Show.”

But this year, with a few months of ratings under his belt, a bit of Colbert’s shine has faded.

Through 12 weeks of the late-night TV season, “The Late Show with Stephen Colbert” is averaging 3.1 million viewers, second to NBC’s “The Tonight Show Starring Jimmy Fallon” (3.9 million viewers) and just ahead of ABC’s “Jimmy Kimmel Live” (2.5 million viewers).

Colbert has succeeded in maintaining Letterman’s second-place ranking, and it was unrealistic to think he could knock off Fallon, who had a year-and-a-half head start (and five years on “Late Night” before that).

And Colbert is still an improvement compared to Letterman’s “Late Show” this time last year. Through the end of November, he’s up 32 percent — while Fallon and Kimmel are down 3 percent and 9 percent, respectively, from their year-ago viewership.

But more alarming for CBS is the widening gap between “The Late Show” and “The Tonight Show.” When Colbert debuted in September, Fallon topped him by 364,000 viewers — a gap that’s grown to 1.5 million viewers for the week of Dec. 14.

When Colbert debuted in September, Fallon topped him by 364,000 viewers — a gap that’s grown to 1.5 million viewers for the week of Dec. 14.

That’s the largest difference yet between the two shows going head-to-head with original episodes.

Colbert, in fact, has beaten Fallon just three times: Sept. 8 (his premiere), Sept. 14 and Sept. 22 (with guest Donald Trump).

“CBS probably does have a reason to worry,” says Billie Gold, VP programming research at media agency Amplifi. “The ratings have gone down and I don’t see them going back up. Fallon has widened his lead. [Colbert is] doing worse than had been expected and he’s skewing a lot older than had been expected.”

Based on same-day viewership, the median age of the “Late Show” viewer 58.5 — the oldest in late night (compared to 57 for “JKL” and 55.2 for “Tonight”).

Undoubtedly — and this is a good thing — some of Colbert’s younger viewers are watching his show via DVR, since “Late Show” has the highest DVR and VOD (video on demand) playback lifts in late night, boosting its under-50 viewership by 29 percent.

So what’s dragging Colbert down? Some blame a roster of brainier guests, like UN Secretary General Ban-Ki Moon, Chief Justice Stephen Breyer and Tesla founder Elon Musk. The host also has not — as of yet — been producing the viral video clips like Fallon (Lip Sync Battle), Kimmel (Mean Tweets) and even James Corden (Carpool Karaoke) that have become an important currency in late night. Colbert’s buzziest moment? A deep, emotional interview with grieving Vice President Joe Biden.

“The people who watch network late-night just want to be entertained,” Gold says. “He had all these political candidates on the first couple of weeks and I think it turned people off … Once you turn off to a show after you’ve given it a sampling, it’s very hard to go back.”

CBS will give Colbert another big promotional push in February, slotting “The Late Show” to air after the Super Bowl, which will easily give Colbert his highest-rated episode to date. But Super Bowl bumps can be fleeting.

Just ask the cast of “New Girl.”