Sonia Khan seemed exactly the kind of young woman likely to prosper under Boris Johnson. A passionate Brexiter who cut her political teeth campaigning for lower taxes, at 27 she was already a veteran of several Whitehall departments, including the Treasury. When he became chancellor, Sajid Javid snapped her up. Then, last week, her career ended in the most brutal of ways; escorted out of No 10 by the police, after being summarily sacked by prime ministerial adviser Dominic Cummings in a manner that her former boss Philip Hammond suggested could win her an employment tribunal case should she choose to bring one.

Worse still, Khan, whose loyalties were seemingly questioned because of her contacts with old friends in the rebellious Hammond’s camp, was the fourth female aide forced out by the new regime. “How someone could think that escorting a young Asian woman out with an armed police officer was good optics beggars belief,” says a female former Tory staffer. She points out that the idea of what one anonymous Tory source called “big thuggy bald blokes picking on girls” is a gift to opposition MPs busy painting Johnson’s wider regime as aggressive, unreconstructed and full of dubious attitudes to women.

Sonia Khan, who was escorted out of Downing Street by police. Photograph: Aaron Chown/PA

Johnson appearing to call the leader of the opposition a “big girl’s blouse” in his first Prime Minister’s Questions on Wednesday does nothing to improve this image – nor did Cummings’ behaviour on Tuesday night, after a vote to wrest control of parliamentary proceedings – widely seen as a humiliating defeat for his strategy – ended with a reportedly boisterous chance encounter between him and Jeremy Corbyn as the latter left for the night. The Labour frontbencher Cat Smith, who was with Corbyn, tweeted that at first she just thought there was “some loud bloke who stunk of booze yelling at us”.

Dominic Cummings, whose sacking of Sonia Khan has caused controversy. Photograph: James Veysey/Rex/Shutterstock

But in an era where behavioural norms and political conventions are being rewritten overnight, the question is whether the charge – fair or unfair – of having a “woman problem” still carries an electoral price for Tory leaders. Will it matter at the ballot boxes that Johnson scrapped an inquiry ordered by Theresa May into Mark Field, the Tory MP filmed manhandling a female protester out of a smart City dinner? Do potential Tory voters care that he compared women wearing burqas to letterboxes, or once suggested the best way to deal with a female colleague’s advice was to “pat her on the bottom and send her on her way”?

When David Cameron told the Labour MP Angela Eagle to “Calm down, dear” in the Commons, the accusations of sexism lasted for days. But have the goalposts changed, in an era now so polarised that Donald Trump can seemingly dismiss an allegation of rape – levelled in the journalist E Jean Carroll’s memoirs – and watch the news cycle move on?

He may now be cast as the cartoon villain of this administration, but when Cummings began work in 2002 for the then Tory leader Iain Duncan Smith, fear was arguably the last thing he inspired in female colleagues. “He couldn’t even manage his own life,” says one. “He used to wash his shirts in the office and hang them up in front of his desk, so you’d have to come through a line of dripping wet shirts to speak to him.” It didn’t last: Cummings left within months, frustrated that shadow cabinet ministers wouldn’t adopt his radical ideas.

It was a different story when Michael Gove, then shadow education secretary, hired him as an adviser in 2007. After Gove moved from opposition to head up the Department for Education in 2010, Cummings was initially kept at arm’s length, but shortly after he formally joined the department in 2011, allegations of him intimidating civil servants began to emerge. They were denied, but it is a pattern some see repeated now in his treatment of Khan and his reported response to fellow special advisers that if they didn’t like his methods they could “fuck off”.

Yet the idea that he picks on women is, say those who have worked with him, untrue. “If he likes you and thinks you’re on side, he’ll work very effectively with you; if he thinks you’re disloyal or useless, he can be extremely unpleasant, but he did that to men and women,” says a longstanding former colleague. “I don’t think it’s a woman problem, I think it’s a ‘people he doesn’t like’ problem.”

His pet hates include perceived stupidity, laziness, or ideas being blocked for seemingly vague reasons, which may explain No 10’s sudden appetite for ripping up unwritten rules and conventions. But Cummings was close to like-minded female members of the Gove set, including the free schools champion Rachel Wolf (married to his old friend, the lobbyist James Frayne) and the adviser Elena Narozanski, now back in No 10 advising on education; he also rated civil servant Victoria Woodcock highly enough to recruit her as Vote Leave’s director of operations, arguing subsequently that, without her, they might not have won.

“Of all the accusations you could hurl at him, being a sexist is actually the least of it,” says Gisela Stuart, the former Labour MP who chaired Vote Leave. “With Dom, you deliver or you don’t deliver; things are good if you do, and not good if you don’t.”

Nor has there ever been a whiff of personal sleaze around Cummings, by all accounts happily married to the journalist Mary Wakefield. The same, however, cannot be said for his boss.

Tory MP Mark Field manhandled a protester at a City dinner. Photograph: Reuters

The plot of Who’s the Daddy?, a bedhopping farce that made its London theatrical debut in 2005, would have been so preposterous as to be incredible had it not been based on a series of real-life scandals that beset the Spectator magazine during a single year of Johnson’s editorship. Co-written by the magazine’s theatre critic Toby Young, it featured not only Johnson’s affair with the columnist Petronella Wyatt – which ended in him being fired from the Tory frontbench for lying about it – but the Spectator publisher Kimberly Quinn’s affair with the cabinet minister David Blunkett and the then deputy editor Rod Liddle’s extramarital affair with a 22-year-old receptionist.

Liddle’s now ex-wife, the broadcaster Rachel Royce, subsequently attacked what she called an “all lads together” culture at the magazine; in Sonia Purnell’s biography Just Boris, she describes famously riotous champagne-fuelled parties “full of young things in short dresses, high heels and lipstick and the men flirting with them and ignoring their wives. I just felt that Boris was running the whole place like a knocking shop.” None of this stopped Johnson being elected mayor in 2008.

His move to London City Hall initially heralded a more statesmanlike phase (although within a year he was to begin an relationship with unpaid arts adviser Helen Macintyre). While most of his senior appointments were male, a recurring theme in his career, he did hire the highly regarded transport expert Isabel Dedring, plus a bright young Oxford graduate named Munira Mirza to cover culture (Mirza now runs his policy unit at No 10).

“I look at him now and he’s got a bunch of lads around him,” says a colleague from that era. “But it was much more balanced in those days. They weren’t the kind of women who would put up with any mischief.” Yet Jennette Arnold, leader of the London Assembly Labour group, publicly complained of his “disrespectful, patronising way at meetings” with female assembly members, while some City Hall staff recall a boys’ school culture of risque jokes and Oxbridge ties.

Throughout his career, Johnson has tended to get on with a certain type of woman; robust, not overly politically correct, dismissive of liberal shibboleths. Mirza famously criticised May’s audit of racial disparities in public services as self-flagellatory, and the new Downing Street recruit Chloe Westley, now heading Johnson’s social media team, argued in a recent article defending the academic Jordan Peterson against charges of misogyny that young women had been “misled by feminists” into thinking they faced discrimination over pay.

His inner circle in Downing Street, meanwhile, remains resolutely male compared with Cameron’s or May’s: this week’s spending review was presented, unusually, by an all-male Treasury ministerial lineup despite concerns about the impact of excluding women from economic policy-making.

As Dr Mary Ann Stephenson, head of the Women’s Budget Group, which analyses the gender impact of economic policy, points out, it has been accepted wisdom for years now that ministerial teams should be mixed in order to better reflect the reality of ordinary lives: “I can’t understand why this lesson needs to be relearned. I remember talking about this in the 90s.” But the emerging hallmark of the Johnson regime is that no conventional wisdom should be taken for granted, perhaps particularly not in the wake of a female PM many female voters did not feel served them well.

The remain campaign fielded an all-female team – Nicola Sturgeon, Angela Eagle and Amber Rudd – to debate with Johnson on TV. Photograph: Matt Frost/AFP/Getty Images

When the remain campaign confirmed it was fielding an all-female team of Amber Rudd, Nicola Sturgeon and Angela Eagle for a televised referendum debate against Boris Johnson, the rationale seemed obvious; they judged female voters to be his achilles heel. Rudd’s jibe that Boris might be fun at a party, but wasn’t the man to drive you home – seen as a deliberate dig at his private life – was deemed the zinger of the night and she was acclaimed a Tory leader-in-waiting. But three years on, she serves in Johnson’s cabinet, awkwardly dodging questions about why she won’t rule out a no-deal Brexit, a reminder that Johnson’s supposed “woman problem” is more complicated than it looks.

His net favourability rating is indeed higher among men than among women, with YouGov reporting a gap of eight points in August. But that may at least partly reflect the fact that women were more likely to vote remain in 2016, and are now more anxious about a no-deal Brexit. Dig deeper, and there is surprisingly little hard evidence that Johnson’s behaviour specifically alienates female voters.

“There’s just not much evidence to suggest that women feel differently from men about him,” says Deborah Mattinson, CEO of Britain Thinks, which will next week publish the findings of focus groups conducted with women in Reading and Liverpool. “The big divide is age – older people like him more than younger ones, and leavers like him more than remainers.” Both sexes question his integrity, she says, but find him charismatic; focus groups generally shrug off the breakdown of his marriage to wife Marina or his new girlfriend Carrie Symonds’ move into No 10. “They’ll say: ‘Look, this is 2019, people don’t necessarily live happily ever.’”

Johnson in 2015 when mayor of London, with Jennette Arnold and members of the British armed forces. Photograph: Dan Kitwood/Getty Images

Such stories may strengthen the resolve of women who are already hostile to the Tories to support opposition parties, which helps to explain why Labour systematically seeks to portray him as a casual misogynist. But it may not actively shift votes in a snap election where Brexit overshadows everything.

Yet if the Khan affair has any lingering significance, it is in uniting the charge – fair or unfair – of sexism, with the idea of a regime playing fast and loose with the rules, rather like a tech bro startup whose culture of being too hip for HR soon turns toxic. “It fits with the narrative that the government doesn’t have to obey the law; they’re going to prorogue parliament if they want, they’re not sure if they’re going to follow employment law, the law is for everyone else and not for them,” says a female former Tory staffer. Some will always thrill to the idea that rules exist to be broken. But not, perhaps, those the rules are made to protect.