Community cohesion has been effectively abandoned by the government, according to a former integration tsar who says the approach is typified by the prime minister comparing Muslim women wearing burkas to “letter boxes”.

Speaking three years after producing a hard-hitting report into ministers’ failure to tackle social cohesion, Dame Louise Casey said the gap between the haves and have-nots had widened, communities had become more divided and Muslim women remained the most marginalised group in the UK.

Casey said that since her 2016 report grievances and prejudice had deepened and toxic views had become more normalised, describing Boris Johnson’s “letter box” Telegraph column last year as “absolutely extraordinary”.

Casey’s report, commissioned by former prime minister David Cameron, advocated initiatives to encourage immigrants to embrace British values, and promote social mixing among young people.

“If you look at the data then and now, little has changed. The brave domestic agenda has been forgotten with universal credit and working-level poverty absolutely symbolic of the country’s dissonance between the rich and poor,” she said.

Her key recommendation of teaching English as a second language had been broadly sidelined, Casey said.

“It needs to be a flood of resources, not just a drop here, a drop there. I would set a target that says in five years’ time we want everybody in the country to be able to speak a common language. We’re all in this together.” Her report found that people from black, Pakistani and Bangladeshi ethnic groups were three times more likely than white British people to be unemployed, with economic inactivity levels unusually high among women from the Pakistani and Bangladeshi communities.

Casey said the situation for many Muslim women had not improved. They still faced a toxic blend of challenges stemming from their gender, their belonging to a community facing hate crime and Islamophobia, as well as sexist, misogynistic and patriarchal behaviour from within their own communities. “It is still so difficult for Muslim women to stand up and be counted,” she said.

Muslim women face a toxic blend of gender and race challenges. Photograph: MBI/Alamy

Casey said young Muslim women should be able to believe that they could emulate the success of BBC presenter Mishal Husain. “When every Asian girl of Muslim heritage has eyes on Mishal Husain, we’re game on. My integration job is over,” said Casey.

Integration expert Fiyaz Mughal said that the government’s response to Casey’s review, which also advocated an “integration oath” to encourage immigrants to embrace British values, had been poor.

Mughal, founder of interfaith group Faith Matters, said: “The government’s response has been too half-hearted and not thought through.”

He said ministerial lethargy over the issue had helped contribute to the societal rancour and increase in hate crimes recorded during the aftermath of the 2016 EU referendum. “It’s part of the reason why the divide in our country opened up so quickly after Brexit, partly because there wasn’t anything strong binding us together, these things can unravel quickly You have to be constantly assisting cohesion,” said Mughal.

Casey said that the left should accept some of the blame for failing to speak up on behalf of Muslim women for fear of being labelled Islamophobic. This had prevented them from challenging misogynistic and patriarchal behaviour in some minority communities, she added.

“The left should be shouting from the rooftops for these women, not being quiet about it. I think people get really confused about being anti-religious. They’re worried about being called racist, but it’s not about faith, it’s about equalities, poverty and deprivation,” said the former civil servant, who is promoting the global homelessness event The Big Sleep Out.

Casey, head of the former rough sleepers’ unit introduced by Tony Blair, also said that the liberal response to the high-profile row over a long campaign to halt LGBT equality messages being taught in a Birmingham school indicated a lack of progress and confidence in defending and encouraging British values.

“I will fight for the right for a Muslim to dress however she wants, but I’ll also fight for the right for Steve to be married to Steve,” she said. “We have to be able to be both – that’s being British, that’s our values and what we fight for – and it’s not an either or.”