A growing proportion of the public believe David Cameron and George Osborne are more capable of managing the economy than their Labour rivals, according to the latest ICM poll for the Guardian.

The proportion of people prepared to back the Tory team for economic competence has soared to 40% from 28% in June. The findings will make grim post-holiday reading for the Labour leader, Ed Miliband, who along with shadow chancellor Ed Balls has seen a much smaller rise in credibility, with 24% of the public preferring them compared with 19% two months ago.

State of the parties July 2013

While the latest voting intentions have Labour ahead of the Tories by three points on 35%, the crucial economic "competence" figures will be a blow to Miliband, who has had a difficult summer following criticisms by backbenchers of the party's lack of direction and leadership.

Monday proved to be another turbulent day for Labour's frontbench when Chris Bryant made a speech partly about the economy which did not go to plan and was forced to climb down over criticisms of the retailers Tesco and Next for using foreign workers.

The two companies accused Labour's immigration spokesman of getting basic facts wrong, forcing him to water down allegations in a keynote speech on immigration before he had delivered it.

The strong increase in backing for Cameron and Osborne to handle the economy comes at a time of early signs of economic recovery in the last month, with growth of 0.6% in the second quarter of 2013.

Labour has recently adapted its message to argue it is only the well off who have been benefiting from increased growth, but the poll findings on "competence" show the party still has much to do to convince voters it can fix the country's economic problems. Such high competence figures for the Tories were last seen in ICM polling in December 2012 when 35% preferred Cameron and Osborne and 24% preferred Miliband and Balls.

On voting intentions, Labour are down one point from last month to 35% and the Tories down four points to 32%. This suggests a return to the balance of opinion observed for much of the last year, with Labour just ahead, following a dead heat between the two main parties in July when the Tories wiped out a seven-point Labour lead from the month before.

Both parties have lost voting share in the last month (Conservatives down four points, Labour down one) with Ukip the main beneficiary, increasing its share by three points, from 7% to 10%. The Liberal Democrats have increased their share by 1% to 14%. However, all of these shifts are relatively small, suggesting a certain level of stability in the UK political scene over the last month.

Worryingly for Labour, the Conservatives are still seen to be better equipped to manage the economy while the public continues to be pessimistic about their own current and future financial situation.

When considering their personal financial circumstances, just 17% of people say they have improved in the last year, compared with 43% who think they have got worse. Thinking ahead to the next 12 months, while a quarter of people think their personal financial situation will improve (24%), more say it will get worse (29%) and 41% think it will stay the same.

Pollsters have warned that few oppositions have won a postwar election without a commanding lead on the economy.

Peter Kellner, a pollster for YouGov, said Labour failed to defeat John Major in 1992 and the Tories subsequently lost 1997 because each party had problems with economic credibility at the time.

"Unless there's some very particular hot topic at the time, it's always the economy, stupid, when it comes to an election," he said.

"The economy is the biggest thing where character matters. The questions is whether the leader up to the job of protecting standards of living. It's a general view of competence, rather than a specific positional view."

Deborah Mattinson, the former pollster to New Labour, said the economy would be "one of the really critical factors and Labour clearly needs to be doing better than it is".

There is a sense of urgency for Labour to start getting a clear message across on how they will affect people's lives, she said. Miliband, who has been on holiday for two weeks while the Tories have dominated the airwaves, will return to the public sphere later this week to comment on unemployment figures. However, he is not likely to make a big keynote speech before this autumn's party conference.

Senior Labour sources say that the row over competence and lack of policies will drift away once parliament returns in September. An adviser close to Miliband said that the current questions over his leadership have been "cooked up" by a handful of backbench MPs and journalists "who have nothing to write about in August".

He also insisted the party is happy with the way Bryant's speech had gone because he was spreading the message about the party's new approach to immigration.

Speaking at a thinktank in Westminster, Bryant said the party now recognised that immigration could have a "negative effect" on British workers.

He said the party has "concerns about immigration, the pace of migration, the undercutting of workers' term and conditions and the effect on the UK labour market". His remarks went even further than Miliband's admission that Labour failed to listen to people's concerns about immigration causing social upheaval.

However, the speech was overshadowed by confusion over his criticism of Tesco and Next for their use of non-UK workers. The Labour MP had initially briefed over the weekend that these high street chains were examples of "unscrupulous employers" that favoured foreign labour over British workers.

He later backtracked during a BBC radio interview, saying these companies are examples of "good employers" that tried hard to recruit UK workers but found it difficult to find staff with the right skills. When he eventually came to make his speech, Bryant significantly retreated from his criticisms, saying he had never intended to call either firm unscrupulous.

His admission that immigration could have a negative impact on British workers came after the Tories had been emphasising their tough policies on immigration in an effort to win back voters from Ukip.

Groups on the left have been concerned that Labour will swing further towards the centre in this climate.

In a letter published in Tuesday's Guardian, the film director Ken Loach has been joined by other prominent figures including the former children's laureate Michael Rosen and the Only Fools and Horses actor Roger Lloyd-Pack to call for a new party of the left.

In the letter, they say: "Not content with signing up to Conservative austerity measures that are dragging Britain's most vulnerable deeper into poverty, Ed Miliband has turned his back on union members who supported his leadership bid."

ICM Research interviewed a random sample of 1,001 adults aged 18+ by telephone on 9-11 August 2013. Interviews were conducted across the country and the results have been weighted to the profile of all adults. ICM is a member of the British Polling Council and abides by its rules.