Empire co-creator Lee Daniels has never been known to mince words, and he did not disappoint Friday at PaleyFest.

The drama's festival debut, which also opened the 2016 PaleyFest Friday at the Dolby Theatre in Hollywood, was a lively one to say the least.

Minutes in, Daniels addressed the criticism from critics and viewers about the pacing of the first half of season two — which has been off the air since December.

"It was growing pains and I think it would have happened to any show," Daniels said. "We were learning. Sometimes you don’t feel like it's enough. That these people — these wonderfully talented actors — are enough and so you bring in this person, ok maybe we'll get more eyes. … "

However, Daniels sounded confident that the show has bounced back for the remaining eight episodes of season two, which kick off Wednesday, March 30. "It's just a process. And at the end of the day, we came back," he said. "It's a learning process. It's really learning about everybody and trying things out. How do you make greatness without testing?"

Part of that, Daniels said, was moving the focus away from glitzy guest stars like Chris Rock, who appeared in the season two premiere, and Alicia Keys. (However, he still called Oscar-winner Sidney Poitier a dream get of his for the show.) "Here's the thing, what I've learned on this journey in television, because I had never done television before, is that we're good enough," Daniels said, pointing to the drama's larger ensemble of series regulars. "Look at them. I want to see them every week."

Terrence Howard, who plays Lucious, came to the writers' defense and described the pressure to tell stories at a "neck-breaking pace" while also trying to "keep it grounded" on the series. "We are a nighttime soap opera," he said. "Somehow we have to find a way to keep it grounded with the speed."

He added, "Every week we hit a hurdle that most people would trip over, but we're able to manage that." However, Howard did admit, "We do have moments where we're like, 'What in the f— is this?'"

During the candid conversation with the show's cast and executive producers — sans co-creator Danny Strong and star Taraji P. Henson — Daniels was also notably open about the creative struggles to keep the bar high on the show's original music. Overseen by executive music producer Timbaland, the show's first soundtrack hit No. 1 last May, coinciding with the season one finale of the drama.

"It's tough. We gotta outdo ourselves and it's hard and that means pushing the actors harder," Daniels said. "Trying not to be corny and trying not to cave into what we think is commercial or what was on the streets and trying to bring you into the streets. That has been a struggle to be honest with you. Making sure that the truth of the streets [is represented] and that we don’t commercialize or sanitize it. … It’s a constant struggle."

Jussie Smollett, who plays Jamal, joked that talking about the "streets" was the way to approach Daniels about particularly over-the-top or possible unbelievable storylines for their characters.

"All you got to do is say, 'The streets are talking,'" Smollett said with a laugh. "He'll be like, 'Honey what do the streets say?'"

In recent weeks, the streets and the mainstream media have been talking about a report in the New York Post saying that Trai Byers, who plays eldest brother Andre, was upset with the creative direction of his character and wanted out for the show's upcoming third season, which was already picked up by Fox. The team took the report in stride.

"When I picked up Page Six in the New York Post and saw that Trai Byers was leaving the show, I said, 'We are a hit,' because they were talking about us," Daniels said. "That's when you know the haters gonna hate that, I was excited."

"I begged Trai to please stay on the show," Howard said facetiously, but clarified, "he was never leaving the show."

Added Byers, "There's not a stitch of truth to it but it does show the popularity of the show. … It's not true, but it really opens the ground to show how media really uses their power to perpetrate negativity built on lies versus positivity, which is built on truth."

Byers rattled off several of his co-stars' charitable works that don't generate quite as many headlines and tweets as the rumors of behind-the-scenes drama, which earlier this season falsely claimed that Howard would be receiving increasingly less screen time.

"We have a responsibility in the media and I hope they're all listening. The truth has to ring louder than the lies."

Daniels once again pressed upon the tightness of the cast, who later broke into song to wish Howard a happy birthday. "We are where we are and ain't nobody gonna take us down. We are a family. We are tight and the haters gonna hate."

Seeing as the panel took place at the very venue where #OscarsSoWhite had occurred just two weeks prior, Daniels also addressed the diversity issues facing Hollywood, albeit semi-reluctantly. "I don’t have time to deal with racism. If I thought, 'Yes, am I delusional to say that there's not racism in America?' Look at Donald Trump," he said, which elicited claps from his cast and the crowd. "The minute I embrace it, it becomes real to me and I don’t have time."

"I don’t wanna hear, 'Woe is me, they ain't treating me right,'" he continued. "You know what? Treat yourself right."

Instead, Daniels focused on doing his show his way and called it an "honor" to work on a series bringing diversity into not only American homes, but homes all around the world where Empire has since become an international hit. Said Daniels, "Words can't describe my happiness."

Empire returns Wednesday, March 30 at 9 p.m. on Fox.