It’s unclear how big of a role its recent controversy played, but A&E’s record-setting reality show “Duck Dynasty” appears to be cooling off in the ratings.

One week after returning to kick off its fifth season with a 3.3 rating in adults 18-49 and 8.5 million viewers overall, “Duck Dynasty” slid to a 2.9 in the demo and 6.65 million total viewers, according to Nielsen. This is the smallest same-night crowd for an original episode of the series since the second-season finale drew 6.45 million in December 2012.

The show, which was embroiled in controversy over the holidays due to derogatory comments about gays and others made by family patriarch Phil Robertson, peaked with its fourth-season premiere last August (5.0 demo rating, 11.77 million viewers overall) — the largest audience for a nonfiction series telecast in cable history.

“Duck Dynasty” also holds the cable reality record in adults 25-54 (6.3 million), topping the mark set by “Trading Spaces” nearly 10 years earlier. The only cable reality show to ever draw more adults 18-49 than last summer’s “Duck Dynasty” premiere was TLC’s “Jon & Kate” in June 2009 (6.5 million).

Last year among all cable series, only AMC’s “The Walking Dead” did better in reaching adults 18-49.

“Duck,” though, seemed to be cooling toward the tail end of last season, falling below 8 million a couple of times before closing with 8.4 million. Last week’s fifth-season premiere ratings were lower than many expected, but above where the show closed last season and nearly on par with its winter bow of 2013.

Among all programs on Wednesday, “Duck Dynasty” ranked behind only Fox’s “American Idol” and ABC’s “Modern Family” in adults 18-49.