Labour leader says right to welfare ‘must be earned’ and admits that the recent success of Ukip is a real danger

Ed Miliband has moved to contain rising panic over Ukip’s growing threat to Labour by pledging a raft of hard-headed measures to ensure that migrants “earn the right” to state benefits and face stiff English language tests before taking up jobs.

Ahead of what promises to be a tense meeting of his parliamentary party in the Commons on Monday, the Labour leader accepts, in an article for the Observer, that Ukip’s successes in recent elections now represent a real danger that Labour cannot ignore.

Miliband stands accused by some in his party of failing to do enough to counter Ukip and of allowing Nigel Farage to amass support in Labour heartlands as well as Tory areas by exploiting worries over immigration.

Concern that Ukip could wipe Labour out in many northern seats next May reached new heights on Friday after the anti-EU party failed by only 617 votes to oust Labour in its previously safe seat of Heywood and Middleton in Greater Manchester. While the Conservatives suffered the humiliation of losing to Ukip in Clacton – where the Tory defector Douglas Carswell became Ukip’s first MP – senior Labour figures say the party has been caught unawares by Ukip, having focused on the cost of living crisis and the NHS, with immigration a lower priority.

One key figure said: “We have had no real response to Ukip, who are now our main opposition in large parts of the north. We just assumed Ukip was the Tories’ problem. The worry is that it is a bush fire spreading everywhere.”

In his first detailed response since Ukip’s byelection surges, Miliband insists that while he will never seek to imitate Farage’s party, Labour has to do more and comprehend why people feel so abandoned that they are turning to Ukip in anger. “We can only do so if we understand many of the people turning to Ukip because of disappointment with Conservative and [past] Labour governments,” he says.

While he claims that his core message that the economic recovery is not benefiting most people has resonance, he accepts that “all this does not automatically translate into support for the Labour party”. He adds that if Labour is to win next May “there is much work to be done”.

Making clear Labour will announce tough new immigration policies in coming weeks, Miliband says that rules limiting access to benefits until migrants have contributed to the state will be based on the principles of “contribution, responsibility, fairness”. As well as stronger border controls and laws to stop “exploitation that has undermined wages of local workers”, Labour will commit to “reforms to ensure those who come here speak English and earn the right to any benefit entitlements”.

Sources close to Miliband made clear that the announcements would go further than Labour’s existing plans to extend the period EU migrants must be in the UK before claiming out-of-work benefits to six months, and to prevent benefits being sent back to families in other countries. It is expected that Labour will impose language tests on migrants to ensure those applying for public sector jobs have a level of proficiency as a condition of being taken on.

Labour will take some comfort from the latest Opinium/Observer poll which shows that the party has a seven-point lead over the Tories. Labour is on 35%, the Tories 28%, and the Lib Dems 9%. But Ukip remains strong on 17%.

Shadow home secretary Yvette Cooper added her voice to those calling for Labour to do more to tackle Ukip: “We should talk more about immigration and the things people are worried about,” she said.

In an interview with the Observer, Mark Reckless, who also defected from the Tories to Ukip and is its candidate in Rochester and Strood, said he expected Ukip to win at least 12 seats at the election: “I think that is now realistic,” he said. The byelection is expected to be called next week, for a date in November. Meanwhile, John Baron, Conservative MP for Basildon and Billericay, refused to rule out following his former Eurosceptic colleagues Reckless and Carswell by joining Ukip. “You should never say never in politics, but the bottom line is my very strong preference is to stay within the Conservative party,” he told BBC2’s Newsnight.

On a visit to Rochester on Saturday, Farage made clear he was not interested in any pact with the Tories: “This party is not a splinter of the Conservative party. This party is its own organic force.”

Neal Lawson, chair of the centre-left pressure group Compass, said the root problem was inequality: “The anxiety so many feel is not the fault of people in their community that are different from them. Even if it was just about votes and not about doing what is right, Labour can never out-Ukip Ukip on immigration.”