I’ve just woken up after a trip to the Conservative Party conference in Birmingham. As a Labour councillor, I was something of an intruder. I went a little red at the hotel reception, mumbling apologetically that I wasn’t “actually a Tory” and feeling shell shocked at the huge numbers of powerful looking men in corporate suits. But as it turned out, one group of Conservatives were far frightening than any other. These are the "nice guys", and they pose a serious electoral threat to Labour.

This group is not a formal alliance, but they are all critical of economic liberalism. They are prepared to challenge the market when it isn’t working for people, and they have a genuine concern for the poor. They are socially conservative, and believe in family, community and tradition. They admit that 1979 brought problems as well as benefits. They are sceptical of big business wielding too much power and stick up for strivers, whether they work in the public or private sector. They believe there is such thing as society. They are, in essence, One Nation Tories.

One man I had barely heard of before the conference actually took my breath away. Guy Opperman, Conservative MP for Hexham, stood up and made a passionate call for apprenticeships, action on low wages, protection for the poor and local banking. He gave up his summer to walk from Sheffield to Scotland, talking to people about why his party was failing in the north, and his speech was clearly rooted in their concerns. I thought woah, if that’s where the Tories are heading, Ed Miliband is going to be left without any clothes.

Jesse Norman MP, a gentle giant who is respected from all sides of the party, is better known for this position. I listened to him explain that growth – even if it does return – is not enough if it only benefits the top. He might not sign up to a living wage, but he does want to challenge corporate governance structures to make a difference. Shaun Bailey, former candidate for Hammersmith, agreed with him. In separate sessions MPs Gavin Barwell and Kris Hopkins warned about “kicking public sector workers” and expressed serious concerns that their party was not perceived to be on side with fireman and doctors. Outside parliament, ConservativeHome founder Tim Montgomerie and funder Lord Ashcroft are fighting to build a party that speaks to blue collar workers as well as white collar corporates.

The good news for Labour is that this half of the party isn’t likely to get anywhere any time soon. Economic liberals like George Osborne and John Redwood are detatched from the concerns of the country. They are to the right of the Daily Mail because they still favour the rich guy over the small guy. They believe they will be able to win over working class and inner city areas because they are in line with the polls on welfare, crime and immigration. But they need a positive vision for the country as well as a negative one. They need to speak about what they love as well as hate, what they will give as well as take. When they refuse to challenge the market or use the state, that’s very hard to do. The polling from working class, northern and inner city areas shows that they simply aren’t cutting through.

“Losing the centre ground is our biggest threat,” says one young Conservative sitting with a group of friends in Carluccio's just outside the conference centre. “The right wingers just think we need to carry on the same way, offer a referendum on Europe and add in some stuff on strugglers and strivers and we’ll be fine.”

“People think we didn’t win (a majority) because we weren’t right wing enough,” his friend chips in, “It’s genuine. Honestly. Do they have no brain?”

If the Tories want to win in 2015, they’ll have to listen to these voices. Returning to growth is not enough if the benefits are only felt by a small number at the top. They will need to steal back the One Nation vision, and that means promoting their nicer half. Such a move would unsettle the left. Luckily for Ed Miliband, that's very unlikely. Economic liberalism still rules the Conservative party, Osborne continues unchallenged, and the centre ground is slowly slipping away.