Washington (CNN) President Barack Obama is used to hecklers stopping him during speeches -- but he draws the line when the audience interrupts him in his own house.

As Obama was speaking at a White House event honoring LGBT Pride Month on Wednesday, an accented voice rang out from the crowd. Obama wasn't amused.

JUST WATCHED Obama disarms heckler's nuclear question Replay More Videos ... MUST WATCH Obama disarms heckler's nuclear question 00:48

"Shame on you," he told his heckler, who was protesting deportations under the Obama administration.

Obama responded, "Listen you're in my house ... it's not respectful."

The interruption persisted, however, and Obama asked for the heckler to be removed from the East Room.