The Labour MP Diane Abbott has urged Ed Miliband not to chase voters who yearn to see fewer "black, brown or foreign-looking people" on British streets, after he pointed out how a growing west African community had changed the nature of Thurrock in Essex.

She said people did not join Labour to see the leader "sounding like a milk-and-water version of Nigel Farage" and Labour risked losing Scottish votes to Alex Salmond if it was seen as "edging towards Ukip".

In an article for the Guardian, she says: "There is a contradiction in telling Ukip voters in Thurrock that you share their pain about west Africans and expecting those same west Africans to vote for the Labour party elsewhere in the country."

Her criticism follows Miliband's speech on immigration in the marginal seat of Thurrock, where Labour lost control of the council last week. He said many Ukip voters loved Britain and did "the right thing" in their communities, as he reached out to disaffected voters who have abandoned mainstream political parties.

Miliband stressed that immigration benefited Britain as a whole but people who had concerns about the issue were not prejudiced, because they had seen Britain change at a rapid pace over recent decades.

In a separate Guardian article, the shadow home secretary, Yvette Cooper, said Labour would not "ape" Ukip by preying on people's fears, but Miliband was right to say "we need to talk more about immigration as well as calling for reform".

She said Britain's attempt to remain an optimistic, outward-looking nation was under threat as public anxiety about immigration grew, employers exploited cheap migrant labour and communities worried about public services.

Abbott took issue with those who advocated talking more about immigration, saying they also appeared to want to "adopt a more aggressively anti-immigrant tone". She said Miliband was well-meaning but she could discern the "steady drumbeat of pressure" to move to the right from some of his inner circle in response to Ukip.

"Some people try to claim that the immigration debate is no longer about race," she said. "But Ed himself gave the game away in his Thurrock speech when he said: 'Immigration has been changing communities fast, including here in Thurrock, a growing west African community, people coming over from eastern Europe.'

"The fact that Ed felt the need to talk about Africans goes to the heart of the policy challenge posed by trying to chase anti-immigrant votes. Because in popular parlance 'immigrant' means anyone that is black, brown or foreign-looking. And what anti-immigrant opinion actually yearns for is to see fewer of these people on their high street."

Abbott – who has repeatedly called on Labour not to pander to anti-immigrant sentiment – said no government could deliver what Ukip voters wanted without quitting the EU, dropping human rights commitments and "encouraging" lawfully settled immigrants to return to their country of origin.

Abbott, who was sacked as a shadow health minister by Miliband last year for failing to show sufficient loyalty, made the comments following research by the British Social Attitudes survey that showed 30% of British people in 2013 would describe themselves as racially prejudiced.

Abbott's criticism came as Lord Mandelson said Miliband needed to get his act together and present better ideas than "crowd-pleasing cost-of-living policies".

Writing for the Spectator on the threat of Ukip, he said the success of past challenger parties such as the SDP and the Greens was "only due to Labour's weakness".

"When Labour got its act together, their appeal faded. The same has to happen now, if Labour wants to become more than a minority administration in 2015.

"Douglas Alexander, the party's election chief, said: 'Labour can win the general election if we take the right steps between now and a year's time.'

"The 'if' has to include fewer crowd-pleasing cost-of-living promises and more counter-intuitive policies."