David Cameron is rushing through a block on European Union migrants' access to benefits from 1 January, the politically fraught date when the remaining work restrictions on Romanians and Bulgarians will be lifted in the UK.

From New Year's Day all jobseekers from the EU will have to wait for three months from their arrival in the UK before they can apply to claim any out of work benefits, Downing Street announced.

The scrambled clampdown betrays the extreme nervousness in Downing Street at the possible reaction of potential Tory voters – and increasingly restive Tory backbenchers – if the public decide ministers have failed to take every measure possible to prevent Romanians and Bulgarians travelling to Britain en masse.

David Cameron said he believed the restrictions would "make the UK a less attractive place for EU migrants who want to come here and try to live off the state".

The prime minister added that he wanted to "send the clear message that whilst Britain is very much open for business, we will not welcome people who don't want to contribute".

He added: "The hardworking British public are rightly concerned that migrants do not come here to exploit our public services and our benefits system."

Yvette Cooper, the shadow home secretary, accused the government of making last-minute changes because it was on the run. She said: "Labour called for these benefit restrictions nine months ago. Yet David Cameron has left it until the very last minute to squeeze this change in.

"Why is the government leaving everything until the last minute and operating in such a chaotic way? Three weeks ago Theresa May told parliament she couldn't restrict benefits in time, now the prime minister says they can."

Nearly 80 Tory backbenchers, including some of its leading lawyers, are backing calls for the UK to defy the EU and retain the tough labour market restrictions on Romanians and Bulgarians for a further five years.

The rebels welcomed Cameron's speeding up of the denial of jobseeker's allowance, but said the lure of Britain for Romanians and Bulgarians was probably highly paid work, and not benefits.

One of the ringleaders of the rebellion, David Ruffley MP, said: "It's not enough to choke off any abuse of benefits because many want to come here to work.

"The minimum wage in Romania is £1 and, for perfectly rational economic reasons, they want to come here to work for £6 an hour. We were told 13,000 Poles were coming under the Labour government and it turned out to be 500,000, putting pressure on public services."

Polls this week show 40% of voters believe 50,000 Romanians and Bulgarians will come to the UK in 2014. Currently, Romanians and Bulgarians can only largely work in the UK on seasonal fruit-picking visas or if they are self-employed.

The backbenchers, not all on the party right, have been thwarted in efforts to stage a last-minute Commons vote on Thursday on the issue, and have instead had to settle for a three-hour debate without a vote. But government whips have been forced to delay bringing back its flagship immigration bill to the Commons knowing that the 80 Tory MPs are supporting an amendment imposing the five-year delay for Romanians and Bulgarians freely coming to work.

That would put the UK on collision course with the EU, and ministers are desperate to avoid another backbench rebellion that just exposes the UK's inability to halt EU migrants. Eight other EU countries are also lifting the restrictions on Bulgarians and Romanians on 1 January.

Nick Clegg, the deputy prime minister, this week supported the tightening of benefits to EU migrants by saying: "It is impossible to defend the core freedom to move from one EU state to another to look for work, unless the public is satisfied this is not the same as a freedom to claim benefit.".

The work and pensions secretary, Iain Duncan Smith, has also changed rules so benefits will only be available to claimants who speak reasonable English.

The Border Agency will also be empowered from 1 January to debar EU migrants from returning to the UK for 12 months if they have been deported for begging, or not being self-sufficient.