It's not a good sign when a party finishes the week with MPs making the same complaints as they did at the start. Yet that is where the Conservatives are now, with the malcontents still fretting that there is no sense of vision or authority from the leadership.

One thing that has changed is that the Tory party now seems rather more noticeably split over how MPs should be behaving. There is the camp who say, either privately or publicly, that Theresa May should go because things are only going to get worse under her leadership. But then there are others who are furious with anyone agitating for a change at the top, whether publicly or privately, because they think it is making everything far worse than it should be.

One senior Tory says: 'The party is behind her - the noise is unhelpful and these colleagues need to realise they are making things worse, not better. No-one could be doing a better job than Theresa May in current circumstances.' Another, one of John Major's 'bastards', plays the row down still further: 'It's actually all a lot more civilised than you journalists wish to make out. There are no fights breaking out in the tea room, we're not losing votes. It's right that there is a lively debate going on – a lot is at stake.'

Charles Walker, vice chairman of the 1922 Committee, has rather blunter advice: 'Just shut the fuck up, everybody. There is an invisible poison pill in our parliamentary system, which is that the ministerial ranks are stocked from the backbenches.

'Over time you get more and more people who haven't been promoted and who think they are being singled out to be overlooked, and you have people who leave the government and become bitter, and as time goes on this feeling takes hold with more MPs. It happens to every government: it happened to Tony Blair, it happened to Margaret Thatcher.'

A number of ministers we spoke to were concerned that the unrest in the party at home was making things much, much harder in Brexit discussions, with May's stature much reduced in the eyes of European politicians, who see an angry party trying to get rid of the woman who is supposed to be leading the negotiations.

One Cabinet minister says: 'I think there was a wobble in support for May amongst Brexiteers but they are starting to figure out that no Theresa May may mean no Brexit.'

But Brexit is a major source of anxiety for a large number of MPs. Steerpike reported earlier this week that the whips have been holding a series of meetings with members of the European Research Group in which MPs were offered cake. One MP present reports that the Victoria sponge that was proffered was 'a truly patriotic cake: everybody should know that a true blue Tory will not be able to resist a Victoria sponge. It would have been a disaster had it been Black Forest Gateau.' However, this Tory and many others are rather less satisfied with what they've hear about whether or not the UK will stay in the customs union. There are threats from Cabinet ministers to walk if the UK does, as was reported in the FT this morning strike a customs union deal for trade in goods, which, as one plotter says, 'would be the end of it'.

There is also the controversy over whether Treasury officials have been fiddling with economic forecasts to make staying in the customs union seem the only economically beneficial option. Jacob Rees-Mogg has continued to double down on this today, tweeting that 'either the Chancellor or his officials are deliberately trying to frustrate Brexit. Ultimately, ministers must take responsibility'. It's worth noting that Philip Hammond is the main target in this particular row, but May also needs to tread carefully: MPs with a very good sense of the mood across the party warn that if the Prime Minister gives Brexit minister Steve Baker a public dressing down for agreeing with Rees-Mogg, then this will cause her serious problems in terms of party discipline.

A number of MPs are threatening to send letters demanding a vote of no confidence to 1922 Committee chair Graham Brady, and are making those threats as they negotiate with the whips, in order to get more out of them. While the chief whip Julian Smith is said to be in 'listening mode', there are some concerns that the whipping operation has actually made things worse over the past couple of months as it is what one minister described as 'too aggressive'. As Coffee House reported earlier in the week, the whips are managing to be both aggressive and passive: they haven't yet mounted a concerted operation to call around their flocks to find out how each MP is feeling.

Others are withholding their letters only temporarily, not because of Brexit, but because of the ongoing lack of vision from May both about Brexit and domestic agenda. One member of the government says: 'She is either on her last life or her second last. The Spectator cover encapsulated perfectly the way a lot of MPs look at this. She needs to change in order to continue but it's not clear she can. Nobody wants to be the person who is responsible for pulling the emergency cord. But conversations are now ongoing about a way to bring this to a point of a new leader.

'After the reshuffle about six of my intake (2015) were talking about letters. Now it's a lot more - and it's the people who are seen as voices of reason. The best cover would be the local elections in May: we need a way to explain to the public why we are doing this so it doesn't look like an act of self-indulgence. It doesn't matter that we are neck and neck in the polls: we need an eight-point lead to have a majority.'

However, the new intake on the whole are still the most cautious about the prospect of a leadership election. 'I struggled to disagree with most of what Nick Boles said on Twitter - we just need to crack on and do something. But internally I see that this will change. I keep saying we need to hold our nerve,' explains a 2017-er. 'The problem is that with Brexit we're beholden to an extreme minority of our colleagues. It's a sad state of affairs that 48 MPs could make something happen that the majority don't want – you'd struggle to find any of the 2015/17 intake who want an early election.'

Most MPs name housing and health as the key areas where May should set out a bold vision if she really wants to survive. There is growing respect for Jeremy Hunt after he decided to stay on at the Department of Health, as Tory MPs believe this shows an admirable commitment to the NHS. They therefore agree with his plea for a long-term financial boost for the NHS, and many have been calling privately for a Royal Commission on the health service.

But there is deep frustration that the government continues to be silent on both topics. 'We're not saying she should go,' says one MP from the 2017 intake. 'It hasn't come to that yet but something has changed. A lot of us are losing patience - but we don't want her to go yet. The government just needs to fucking do something. Where are the new ministers? They should be sending them on the airwaves. Theresa May should focus on Brexit but that doesn't mean the rest of the government has to.'

For the time being, May will benefit from there being such a deep split in the party between those who think she has to go or the party will suffer and those who think that pushing her to go will damage the party even more. But it's clear from our conversations with Conservative MPs today that the silence of many MPs is conditional on her getting on with restoring her authority and setting out a vision. For another Prime Minister, that might seem like a reasonable demand. Many MPs, though, fear that for May this has become impossible.