It is the end of yet another momentous week in parliament and Harriet Harman’s kitchen is a picture of cheery domestic chaos. Her two beloved cats are climbing the curtains, there is a pile of washing waiting and Holly, her young granddaughter, who is currently staying, toddles in and out. Harman apologises for the mess, but is clearly rather enjoying having a highchair at the table again.

It is 37 years since she arrived in Westminster, heavily pregnant and feeling like a “fish out of water”, as she described it. Male Tory MPs would shout “stupid cow” at her across the chamber, while the things she longed to debate – such as her south London constituents’ struggle to find childcare while they worked, or feminist reforms she had taken up as a young lawyer – were deemed trivial. At her lowest ebb – contracting pneumonia while exhausted from juggling an opposition frontbench job and small children – she considered quitting. But instead she rose to the cabinet under Tony Blair, served as caretaker Labour leader after Ed Miliband left and is now running to succeed John Bercow as Speaker of the House of Commons when he retires next month, on a platform of putting parliament back in touch with the public that it serves.

Leafing through her notes from the mini listening tour she has undertaken, asking people around the country what they think of parliament, underlines how difficult that has become. Westminster politics is seen as aggressive, entitled, phoney and unprofessional, a braying bear pit hopelessly out of step with modern workplaces, where respect and empathy are increasingly valued. “One woman said: ‘I’m a trade union negotiator. I’d get nowhere if I walked into a negotiation and behaved like that,’” Harman recalls. “One of the things that’s been very striking is that people think the anger that’s displayed in parliament is an artefact, that it’s basically fake, playing up to the cameras. So if anybody’s under the impression that their anger is regarded as inspiring by people outside – people don’t buy it.” And that hints at something more complicated than the crude “people v parliament” narrative pushed by Downing Street, pitting furious leave voters against a supposedly obstructive elite.

Harman’s research suggests the angry, uncompromising stances that many Brexiters (and, arguably, parts of the left) see as connoting passion or ideological purity don’t always come across that way to voters, many of whom see squabbling and division merely as proof of impotence. Resorting to shouting is seen as “the ultimate failure of an institution”, says Harman. “In the outside world, people disagree about Brexit but they get along.” If she’s right, then politicians who believe they are channelling the public mood by ratcheting up their language may be in for a shock come a general election.

We have, of course, been here before. David Cameron vowed to end “Punch and Judy politics” and Jeremy Corbyn promised a “kinder, gentler” form of it, yet neither delivered. Some argue that the adversarial nature of politics means the temperature can’t be lowered, but Harman disagrees. “The overwhelming majority of MPs don’t behave like that in their [private] life and know that it’s not the right way to present our work to the public, but it’s been impossible to stop it. We have got to enable parliament to decide for itself that it’s going to change.” Change can’t simply be imposed, she says, but “the Speaker can facilitate parliament actually looking at itself in the mirror and deciding: ‘Do we want to go on like this?’ If the answer is: ‘No, we don’t’, then the Speaker has got the mandate to change it. The Speaker has got all sorts of controls over how they select people to speak, or deal with those who interrupt …”

She insists that doesn’t necessarily mean a bland or neutered politics, citing the 2003 debate over the Iraq war, during which the foreign secretary, Robin Cook, resigned. “Nobody felt that there weren’t passionate beliefs on all sides, nobody felt that was not a debate of the utmost seriousness. There were very big divisions in the country and in parliament. Yet the debates did not descend into shouting, abuse and finger-jabbing. It’s not about dumbing down parliament or fudging over differences, it’s about respecting what the people think about their parliament. We can’t ignore that because, if people have contempt for their parliament, that’s undermining parliamentary democracy and creates a vacuum.”

Parliament rarely looks more out of touch than during the state opening, when the Queen arrives in a horse-drawn carriage to a flurry of bowing, but Harman isn’t advocating reducing the pomp. “I think people recognise that for what it is, part of our culture and heritage and tradition.” And while she backs changing summer recess dates to reflect Scottish school holidays, which would allow Scottish MPs some time away from Westminster while their children are off, more broadly she insists that it’s not for the Speaker to dictate how parliament should modernise, but merely to facilitate what MPs want.

Still, the longest-serving female MP must think it is ridiculous that MPs have no formal right to maternity leave from their constituency; and that Labour’s Stella Creasy has had to have a public battle to get locum cover for when she gives birth this autumn. Harman says that attitudes have shifted thanks to the influx not only of women MPs but of men who expect time off when they become fathers. “I remember, back in the day, one of my fellow MPs got up and said: ‘On a point of order, Mr Chairman, can I just ask the committee to take note that my wife has just had a baby?’ I was thinking: ‘So why are you here?’ but everyone was like ‘hear hear’ and then we carried on with the committee. That’s not the aspiration of men having young families now in the House of Commons or outside.” Yet as she concedes, it was just last year that pregnant MPs won the right to vote by proxy, and that was only after Labour’s Tulip Siddiq arrived in a wheelchair on the day she was due to give birth. “You mustn’t ram raid things through, but really, it’s kind of glacial, the pace of change.”

Worse still, some now fear progress shifting into reverse, thanks to a tide of threats and abuse disproportionately targeted at female, black and gay politicians and seemingly designed to intimidate them out of public life.

“Women speaking up is like a red rag to the misogynist bull,” sighs Harman. “That’s what the response to the significant advance of women in parliament has been; a significant backlash together with social media enabling anonymous threats and abuse.” She doesn’t see it putting Labour women off standing for parliament, but worries that “people stand down earlier than they might because of that”.

And latterly, such abuse hasn’t always come from outside the party. Female and ethnic minority MPs have been disproportionately targeted so far in the trigger ballots allowing local party members to seek to replace their candidates, just as women warned would happen when Corbyn changed the rules. Lucy Powell, the Manchester Central MP, has argued the process has become in parts about “trying to put a lid on opinionated, outspoken women”; in South Shields, Emma Lewell-Buck has raised allegations of bullying and harassment within the local party.

Harman argues the threshold for deselection was set too low. A reselection contest can now be triggered if 30% of constituency branches want one, meaning in practice that an MP’s fate can be dictated by a handful of aggrieved people – and needs reviewing. But if it were just about a technically flawed process, men would surely be affected as much as women? “Certainly, some women have said: ‘I got selected on an all-women shortlist, there were some people in the party who have never accepted there being an all-women shortlist, have always resented me and have seen this as their opportunity not necessarily to get rid of me but to give me a hard time,’” she says. While trigger ballots are still at an early stage, she cautions: “Obviously, the process is flawed and whatever the intention of it was, it’s not fulfilling the intention.”

Like many Labour MPs, Harman expressed dismay when Louise Ellman, the Jewish Labour MP for Liverpool Riverside who had faced a long and vitriolic campaign to oust her, resigned the whip, saying Labour was “no longer a safe place for Jews”. (Ellman’s critics accuse her of jumping before she could be deselected. That isn’t true, Harman insists; she was tipped to survive the trigger ballot process.) So what, apart from issuing anguished statements of regret, are MPs like Harman going to do about tackling antisemitism within Labour?

“We do need the Equalities and Human Rights Commission [EHRC, now running a formal inquiry into antisemitism in the party] to bring forward its report ASAP because that will provide the imperative for change that’s needed, and end the situation where people are accusing those who have been victims of antisemitism of confected concerns,” says Harman. “I think [publication of the EHRC report] is the point at which we will [enact] the long-overdue change.”

Shouldn’t Corbyn take some personal responsibility for what is erupting within his party? Harman chooses her words carefully: “I think the thing about leadership is that it’s your responsibility not just to have the right position, but to ensure that it’s the position of the party as a whole. That’s not just about saying: ‘I’m totally opposed to antisemitism, I do not condone it, I’ve never engaged in it.’ Leadership is about making sure that wherever it shows itself, it’s absolutely stamped down on. I think that has been the problem.

“It’s important to respect and support the autonomy of the members and that’s a role of leadership, but it’s also a role of leadership that if members are doing something which is egregiously wrong, you clamp down on it. You have got to be tough and directive to be an effective leader as well as supportive of your grassroots.”

Even such veiled criticism is significant because, unlike many of her old political allies, Harman has remained publicly loyal to each successive Labour leader, whatever her private misgivings. (She declined to serve on Corbyn’s frontbench, arguing that she had had her turn, but her husband, the MP Jack Dromey, does.) She takes a broadly stoical view of the current turmoil, having lived through Neil Kinnock’s struggle with the hard left when Labour councillors going into meetings were spat on by local activists. “My constituency party had to be suspended in the 80s because there were fights in general committee meetings. Someone pulled a gun at a meeting and I was like: ‘This has really gone beyond ...’ and they said: ‘No, it was only an imitation gun,’” she recalls. “I’m not saying that what’s happening now should be discounted, but the party found its equilibrium after that turmoil and it will do again after all this.”

Yet in the Speaker’s job she has perhaps identified a role that would lift her out of the fray, obliging her to become a neutral servant of parliament during a constitutional struggle over Brexit that transcends party lines. While critics argue she’s too establishment for what became, under Bercow, a reforming and even activist role, she clearly finds the charge bizarre, having spent her life being criticised for being too radical in her feminism, only to find her causes eventually becoming mainstream.

“I’ve certainly been there a long time and stuck it out – if that’s what you call establishment, surviving, definitely ... But if you’ve been an outsider, a very young woman in a parliament of older men, you never ever forget the contempt with which your views were treated and you never ever forget your complete frustration that other people’s views could be accepted with reverence.”

Does she feel sympathy then for Theresa May, as Tory Brexiters who snubbed her deal leap on board her male successor’s version? For the first time, Harman falters. “You stand or fall by what you can achieve,” she says eventually. “I wouldn’t want to, because she’s a woman, make excuses ... I mean, she was the prime minister, for heaven’s sake. If you can’t take responsibility for your own successes and failures when you are prime minister, then when can you?”

She is more comfortable agreeing with the shadow chancellor, John McDonnell, that the next Labour leader should be a woman, having ducked out of running for the job herself in the past. “I do remember people saying, when I was acting leader: ‘Why aren’t you going for the real thing?’ but I sort of treated it as if they were being polite. I wasn’t quite convinced they really meant it.”

Instead, at 69, she sees herself as part of a wave of older women refusing to leave the workplace quietly. “There are a lot of women who stepped back when their kids were younger and they’re now ready to step back up, but find that, too often, people think: ‘Oh, now you’re past it,’” she says. “One minute you’re too young, the next you’re too old, whereas with men they’re always in their prime.”

Yet with age, she thinks, comes at least one consolation. “If people have just got used to you, they’ll deal with what you say at face value, and that gives you opportunities to do things that you wouldn’t have been able to before ... A lot of them were scared of me when I first arrived; for good or ill, I don’t frighten them any more.” She grins. “I’ve obviously got to up my game.”