Four days after Mayor Bill de Blasio called on Manhattan Board of Elections Commissioner Alan Schulkin to resign over his claims of voter fraud he said partially stem from his municipal identification program, Schulkin said he’s staying put—and that the mayor “doesn’t control” the board.

Conservative nonprofit Project Veritas secretly recorded Schulkin, a Democrat, saying that de Blasio’s IDNYC initiative initiative does not properly vet applicants and enables them to commit election fraud. He also said people are unaware that in minority neighborhoods, organizations “bus people around to vote” to different polling sites and accused Muslim women of using their burqas to hide their identities. He also called for a voter I.D. law.

Last week, de Blasio called Schulkin’s assertions about voter fraud—which he dismissed as an urban legend—and the municipal identification program “crazy” and proof that he is not fit for his position.

“I have no intent to resign,” Schulkin told the Observer today, following a BOE meeting in Manhattan. “The mayor doesn’t control the Board of Elections.”

During the meeting, Susan Lerner, executive director of good government group Common Cause, said her organization and others were surprised to hear of Schulkin’s comments that he made given his position. She said there needs to be clarification and serious consequences for being on the record, even if he described the individual as someone who was heckling him as a representative of a particular organization. He had to have known that his comments were not going to be held privately, she said.

“Your comments about the—even potential—fraud seem really unwarranted and particularly in this super-charged election season we are suffering through,” Lerner said, “to read that a member of the New York City board feels that people who are Chinese, people who wear particular religious garb might be more inclined to commit voter fraud is really shocking and the explanations that we have read really does not tally.”

Schulkin said he was not planning to speak on the record about the incident but that he would address it because Lerner brought it up. He noted the incident happened at a Christmas party last year—well before Donald Trump began tossing around cavalier claims the November election will be “rigged”—and that he did not know who the woman was.

He said she started talking to him but admitted that he should not have spoken with her and insisted that the comments he made in the video do not represent his views, in contrast to statements last week in which he stood by his caught-on-camera call for requiring voters to show a photo I.D.

He also said de Blasio was wrong to characterize his statement as a public statement, saying that it was a private statement that he regrets because he was trying to get away from the woman “and I was agreeing with her when I shouldn’t have been.”

“Anybody who knows me knows I have no prejudice in my heart,” Schulkin said. “I never intended to hurt anybody. I’ve spoken to a number of legislators and they all agree they know me very well and they didn’t take it as an insult.”

Schulkin also acknowledged he was initially skeptical of the IDNYC program but said the woman did not identify herself. When Lerner characterized his response to her concerns as insufficient given “that there are significant concerns,” Frederic Umane, secretary, appeared to defend his colleague.

“Isn’t part of what your function with election protection is to make sure that there’s no fraud going at the ballot?” Umane said. “Isn’t that part of what your observers are looking for while they’re there, to make sure that there is no fraud?”

Lerner insisted instances of in-person voter fraud in the United States are rare, and argued that it was important to eliminate—not create—obstacles at the polls.

“We’re there to be an assistant to the board and to voters,” she said. “But again, the concern is for these sort of reported comments that I think the public has a legitimate concern. That’s all that I’m saying.”

Schulkin, for his part, continued to defend himself.

“I know I shouldn’t say anything, but the people I’ve talked to tell me they don’t and I’ve talked to a number of people from various ethnic groups, elected officials and they told me they know who I am, plain and simple,” he said.

The mayor’s office apparently found the commissioner’s comments insufficient, and persisted in its demands he abdicate his seat.

“The allegations made by Mr. Schulkin are baseless and inaccurate, not to mention racist. New Yorkers deserve public officials who are committed to defending democracy, not undermining it,” said de Blasio spokesman Austin Finan. “Mr. Schulkin has proven himself unfit for his job. He should step down.”

The City Council votes to appoint BOE commissioners like Schulkin, and the county political machines nominate them.

Disclosure: Donald Trump is the father-in-law of Jared Kushner, the publisher of Observer Media.