Ed Balls's personality would be a serious obstacle to any future Lib-Lab coalition, according to senior members of the Liberal Democrats, who describe the prospect of working with him as intolerable.

The shadow chancellor may have to accept a lesser role in any future coalition as a price for the Liberal Democrats partnering Labour after the next general election.

Plans to sideline Balls have been discussed at the highest levels of the Lib Dems, where there is growing unease at the prospect of working with Labour in the event of another hung parliament. One scenario envisions a deal with Ed Miliband involving the resignation of Nick Clegg in return for preventing Balls moving into the Treasury should Labour emerge as the largest party.

It is understood that business secretary Vince Cable is particularly keen on taking on the role of chancellor should there be a coalition with Labour.

A senior figure close to the Lib Dems leadership said: "With the Conservatives, it is a straightforwardbusiness relationship. Whatever the public perception, George Osborne is a fairly straightforward person to deal with, more so even than David Cameron.

"But Balls is Gordon Brown without any of the beliefs and passion. He is a bullying figure and working with him would be intolerable. Personalities matter, you can't take them out of it. Ideologically, there may not be too many issues but the Labour leadership as it stands is toxic. Balls could not be trusted."

Balls, a special adviser to Brown when he was chancellor, is regarded by many at the top of the Lib Dems as a tribalist who would be incapable of working with a rival party. He was one of Brown's chief lieutenants during the worst times in the relationship between Brown and Tony Blair, and he has been forced to deny involvement in a "brutal" attempt to oust the then Labour prime minister in 2005.

In a sign of the concerns, the Lib Dem deputy leader Simon Hughes on Sunday will speak at a fringe event at Labour's party conference in Manchester where he will publicly stress his frustration at the tribalism of those at the top of Labour.

In what will be seen as a warning to Miliband that the Liberal Democrats would not automatically join Labour, Hughes will say: "I am proud of many of the things which we have achieved, but what has been frustrating has been that too many people in the Labour party have let petty tribalism get in the way of progress. I understand that many people in the Labour party are upset that fellow travellers on the left have joined a coalition with the Tories, but frankly they need to get over it. We are doing what we need to do, no more and no less."

He will add: "None of this means that Labour and the Liberal Democrats cannot work together – but if coalition is about putting aside your differences in order to work with others then this is something which Labour are showing that they are just not ready for yet." The manoeuvring against Balls will be a blow to the shadow chancellor who has stressed his eagerness to work with the Liberal Democrats.

On the BBC's Andrew Marr Show shortly before the party conference season, he made public overtures to Cable, a fellow guest on the programme. Balls said that while Labour wanted to win outright at the next election in 2015, they would be prepared to enter into a coalition with the Liberal Democrats if they ditched Nick Clegg as leader.

In response, Cable said he was "not embarrassed to describe myself as someone on the centre left" and defended his links to senior Labour figures, including the sharing of text message conversations with Miliband.