Labour MP Khalid Mahmood has called on the embattled trustees of Birmingham Central Mosque to "consider their positions" after the mosque chairman was accused of dismissing concerns about domestic violence.

Muhammad Afzal, a Labour councillor and Chair of Birmingham Central Mosque, was forced to resign as Birmingham's Lord Mayor elect after audio recordings emerged of him calling Prime Minister David Cameron an "Islamophobe". He also called for a boycott of Prevent, the Government's counter-terrorism programme, and said that plans to introduce safeguarding measures for supplementary schools, including madrassas and yeshivas, were "racist".

He also denied that significant numbers of British Muslims were joining the Islamic State.

Afzal was recorded saying: "I think the Prime Minister is an Islamophobe, he never talks about anything else but Muslim extremism.

"It is ridiculous that the government is saying Muslims are becoming radicalised. David Cameron says 500 people have gone to Syria to become radicalised, but where is the evidence? And out of a population of three million Muslims in the UK, what kind of percentage is that?

"The government wants to inspect madrassas, which is racist. Ofsted are losing their neutrality and what we'll find is that madrassas will be closing down under this legislation which is totally ridiculous."

Shortly after he had resigned from his position as Lord Mayor elect, Afzal became embroiled in another controversy after the Muslim Women's Network (MWNUK) accused him of telling them that "forced marriage was no longer a problem" and claiming that more men than women were victims of domestic violence.

During a meeting Afzal reportedly asked the campaigners, "women these days are strong and educated, how can they be forced into marriage?"

Afzal denied the validity of official figures about the extent of forced marriage – particularly in the West Midlands. He reportedly said the figures were "exaggerated."

According to the MWNUK, he added that any event on forced marriage held with the mosque as a partner would "send a wrong message to the community that forced marriage was a problem when actually it is not."

When the discussion turned to domestic violence within Muslim communities, the MWNUK say that Afzal stated "domestic violence was happening mainly in the Christian community because they get drunk."

The campaigners said that Muslim women had been murdered as a result of domestic violence, with many cases in the West Midlands alone, but he denied hearing of any such cases – despite the mosque running a divorce service where "domestic violence is regularly raised as a reason for divorce," as MWNUK put it.

Khalid Mahmood MP has now argued that Birmingham Central Mosque can only say they are "not aware" of forced marriage and domestic violence "because of the fact that virtually all their committee members are men."

"There should be more women on the committee to highlight some of these issues," he added.

Campaigners are now lodging a complaint with the Charities Commission because the mosque has 39 male trustees and not a single female one.