The man whom New Jersey Gov. Chris Christie (R) told to "sit down and shut up" is not backing down from his fight with the notoriously confrontational governor.

Speaking Tuesday at an event marking the second anniversary of Superstorm Sandy, Christie was repeatedly heckled by local activist Jim Keady, who was holding a sign imploring the governor to "finish the job" of helping the Jersey Shore recover. Christie grew frustrated and fired back at Keady.

“I’ve been here when the cameras aren’t here, buddy, and done the work. I'm glad you had your day to show off, but we're the ones who are here to actually do the work,” Christie said. "So listen, you want to have the conversation later, I’m happy to have it, buddy. But until that time, sit down and shut up.”

Watch video of the exchange above.

Keady, a former Asbury Park councilman, later told MSNBC's Chris Hayes that he knew what he was getting into when he decided to confront Christie at the Belmar, New Jersey, event.

"I had a feeling the governor was going to respond in the way that he did," Keady said. "I think it's unfortunate. I think he shows so little respect to the citizens of New Jersey and to the office of the governorship when he acts the way that he did today."

Keady accused Christie's administration of being slow to allocate the $1.1 billion in recovery funds set aside for rebuilding and repairing New Jersey homes, noting that just a portion of the funds have been used so far.

Christie acknowledged those frustrations during the Tuesday rally, insisting his administration is "doing the best that we can."

In a separate interview with Bloomberg Politics, Keady said he would continue to press Christie on the recovery funds.