(CNN) Steve Harvey has set boundaries and is standing by them.

Harvey came under fire Wednesday after a strongly worded memo sent to the staff of his daytime talk show was leaked by a blogger Robert Feder.

In the note, which seems to have been sent ahead of the current fifth season of his talk show "Steve Harvey," the host asks that there be no attempts to meet with him while he's in his dressing room, in the makeup chair, or in the hallway -- "unless I ask to speak with you directly."

He also insisted that all those attempting to communicate with him "schedule an appointment."

"IF YOU OPEN MY DOOR, EXPECT TO BE REMOVED," he wrote, in all caps. "My security team will stop everyone from standing at my door who have the intent to see or speak to me."

Read More