Former chancellor Kenneth Clarke has cast serious doubt on the underlying strength of the British economy, saying there is a "long, long way" to go before it is competitive enough to deliver sustainable growth and compete with emerging powers such as China and Brazil.

In an interview with the Observer to mark the end of an illustrious career that has spanned more than 40 years in government, Clarke says Britain has to break out of the "ludicrous cycle" of house price booms followed by crashes, and must focus on creating a productive manufacturing base in tandem with vibrant financial and other service industries.

Admitting to tensions with David Cameron since the 2010 election – and voicing frustration at attempts by the Tory press office to keep him off TV and radio – Clarke insists that on economic policy he is completely "on message" with the government.

However, in stark contrast to recent bullish messages about growth and job creation from the prime minister and chancellor, Clarke stresses that the economy is "fragile", vulnerable to shocks, and still lacks the strong productive base necessary to compete long-term in global markets.

"It's not firmly enough rooted on a proper balance between manufacturing and a wide range of services and financial services," he says. "I mean, we have this mystery of why we can't get productivity to start rising again."

Clarke says the coalition has saved the country from economic calamity by reining in spending, and declares himself a "great fan" of George Osborne.

But he fights shy of triumphalism, making it clear that "a bit of cyclical upswing" in recent months should delude no one into believing the job anywhere near done.

While he believes that the Conservatives are likely to be the largest party at the next election, and should use their economic record so far as the trump card against Labour, he doubts that they can win outright.

Looking back across a career that included spells as health secretary, education secretary, chancellor of the exchequer and justice secretary, and three unsuccessful campaigns to lead the Conservative party, he says that the Tories' internal splits over Europe are wider than ever, and his party is no longer the disciplined fighting force that it was when he first became an MP, in 1970.

"I belong to a Conservative party that used to be able to win elections … it was the self-discipline of the party that was extraordinary," he says. Now, however, he sees a lot of Tory MPs who regard being members of the governing party as "a secondary consideration" to banging the anti-EU drum. Winning a majority in May next year is a "mountain to climb", he says.

Clarke, now 74, stepped down from the Cabinet on Tuesday in Cameron's reshuffle, but says he will stand again at next year's election, partly in order to fight to keep Britain in the EU in the event of an in/out referendum, which Cameron has promised to hold by the end of 2017.

While he avoids any direct criticism of the prime minister, he says he thinks his own appointment after the 2010 election as justice secretary "turned out to be a slight mistake" because, as someone on the left of the party, "unfortunately my views didn't coincide with No 10's … David and I didn't fall out exactly, [but] there was constant friction."

He also accuses the Tory press office of trying to keep him off the media and says he had a "great row with them" a few months ago when they told the BBC he was ill and couldn't go on Question Time, when, in fact, he was in rude health and raring to go. When he rang the producer, he says, "she said 'I'm told you're ill'. After that I got even more freelance."

A vote to leave the EU would, Clarke says, leave Cameron and future PMs as diminished figures on the world stage, and the UK unable to shape economic rules that would directly affect this country's economic future.

"Being prime minister will be a less important job if we leave the European Union. You'd become a minor player in all kinds of situations."

Deriding the view of Eurosceptics that the UK could forge its own bilateral agreements outside the EU, he says: "The idea that the United Kingdom could negotiate a serious trade deal with the United States on its own … is ridiculous."

Housing bubbles followed by collapses remain a British disease that has now spread outside London. "I very much hope the British are going to get out of this ludicrous cycle of ridiculous housing booms followed by housing crashes," he says, adding that Bank of England governor Mark Carney is now addressing the problem.

The focus for the next five years must be on pushing on with reforms to create higher levels of skills, more training, better education, more research and development and "boosting science which is difficult with the British who aren't very interested in science. All that. We have got a long, long way to go."

Echoing one of the central messages of Labour leader Ed Miliband, he says: "We don't want to be a low-skills, low-productivity, long-hours economy."

He adds: "We're only halfway through … We've saved the country from calamity but we've got a long way to go before we get a competitive economy, with sustainable levels of growth and all that."

• Kenneth Clarke: I had a lot of views, but they didn't coincide with No 10's