The Tories’ failure to unite the country has led to a “toxic” debate in politics, where voters feel excluded and do not trust politicians, Chuka Umunna says today.

The former shadow Business Secretary criticised Iain Duncan Smith, who resigned from his cabinet position on Friday evening, saying Smith had a “right-wing agenda” which failed to deliver social cohesion.

“In his Andrew Marr interview yesterday Iain Duncan Smith spoke of a One Nation Britain. There will be those who will dismiss it as a soundbite, or worse, an offensive label for the right-wing agenda the former welfare secretary has been pursuing since 2010.

“There is a reason why it is a mantle both Labour and the Conservatives have sought to own and rightly so. It is the expression that sums up the national unity we saw at moments such as the Olympics.

“If we’re honest, it’s an aspiration and not a reality in Britain today. We are failing to live up to the promise of Olympic Britain which is why we need a new focus on how we can build a more integrated Britain.”

Umunna’s comments follow new data from The Challenge that shows a growing gap between rich and poor people’s experiences of London. The majority of poor Londoners do not feel “at home” in the capital, while richer Londoners feel the opposite. Richer residents feel safer, engage in their communities more and have a stronger sense of belonging.

Umunna, MP for Streatham, spoke about the widening gulf between classes and the unfair stereotypes pushed on poor people.

“Rising inequality is one of the most pressing problems facing our nation, and is compounded by the growing segregation of different class groups.

“It’s no wonder we’ve wound up with TV programmes like Benefits Street – produced by well paid people who are not on benefits – which treat people who claim unemployment benefit like a different species.

He said Labour needed to break down barriers to integration, particularly to groups who feel threatened by immigration.

“Labour has rightly argued that immigration has brought real economic benefits, but this is an accountant’s answer to a question which goes to the heart of how people feel about modern Britain.

“We need a concerted drive to break down barriers to integration and provide much better support to help immigrants not only learn English and understand British values, but also meet and mix and build relationships with Britons from all backgrounds – integration, afterall, is a two way street.”

Umunna used the speech to announce a new All Party Parliamentary Group on social integration, which will explore how to build a more integrated Britain. They will focus creating shared experiences and social ties which enable people to develop shared identities.