The conventional wisdom is that the Conservatives are good at elections. Last month they failed spectacularly. But do recriminations about negative tactics mask deeper problems for a party that hasn’t won convincingly since the Thatcher era?

In September 2015, a few months after the Conservatives had won that year’s general election, more comfortably than even their most optimistic supporters had hoped, a veteran Tory politician and journalist was waiting to appear on a BBC radio show. Still smiling about the election, he was in expansive mood. The party’s targeting of voters had become so precise, he told me, thanks to the latest marketing software, that it would take Labour many years to catch up.

During this year’s general election, as in 2015, Tory activists across Britain were supplied with computer-generated lists of amenable voters by Conservative campaign headquarters in London. But this time, many canvassers got a shock when they knocked on doors. “The data was only 65% accurate,” says a local Tory organiser who has worked in the party’s heartlands in southern England for decades. “In the marginals, it was less than 50%.” In some cases, canvassers were accidentally sent to the addresses of activists for rival parties. The organiser says: “I despair of our national campaign.”

Facebook Twitter Pinterest A voter in Abingdon, Oxfordshire, confronts Theresa May over disability benefit cuts during the 2017 general election campaign. Photograph: Stefan Rousseau/PA

The idea that the Conservatives are good at general elections is one of the least-challenged conventional wisdoms of British politics. A large and loyal core vote; copious party funds; flexible beliefs; disciplined campaign messaging; slick presentation; an utter lack of squeamishness about going negative; the relentless support of most of the press; an ability to portray opponents as alien to Britain, to game the electoral system and to set the terms of political debate; and the pervasive belief that the Tories, whether you like them or not, are somehow the natural party of government – these ingredients, it has been said for decades, add up to a political magic formula, one of the most potent in any democracy.

Yet in the 2017 election, it failed spectacularly. An anticipated landslide, imagined by some Tories and their media cheerleaders as the largest since their near-annihilation of Labour in 1931, turned in a few weeks into no majority at all – and a deep sense of Tory foreboding about the underlying direction of British politics, and about the next general election, widely expected in Westminster to come suddenly and soon.

“It was the worst campaign I’ve ever seen,” says Tim Bell, the advertising and PR man who played a central role in Margaret Thatcher’s election victories in the 70s and 80s. Ever since the launch of the disastrous Conservative manifesto in May, Tory recriminations about the election have grown steadily louder: against Theresa May, for her frozen, almost unwatchable campaigning style, against her (now former) advisers Fiona Hill and Nick Timothy for their abrasiveness and over-confidence, and against Lynton Crosby, the much-vaunted Australian political strategist. “The campaign was entirely negative,” says Bell. “That’s Lynton Crosby’s style. He’s a one-trick pony.”

But for Tories to blame their bad election on a few individuals is too convenient. The uncomfortable truth for Conservatives, rarely acknowledged, is that their supposedly all-conquering party has not won a substantial general election victory since Thatcher’s final landslide 30 years ago. Since then, the party has sneaked a small majority in 1992, suffered heavy defeats in 1997, 2001 and 2005, been frustrated by a hung parliament in 2010, eked out another small majority in 2015, and been disappointed by another hung parliament in 2017. Labour’s internal battles and erratic electoral performance over the same period, always eagerly covered by the media, have distracted attention from this long Tory struggle. To adapt a famous Tory slogan from the 70s, Conservatism isn’t working at elections.



Facebook Twitter Pinterest May’s former advisers Nick Timothy and Fiona Hill. Photograph: Chris J Ratcliffe/Getty Images

Tim Bale, a historian of the party and professor of politics at the University of London, argues that the Tories have gradually stopped presenting voters with an appealing vision of Britain. “In the 80s, they had a good balance of negativity and positivity,” he says. This meant exploiting the flaws of Labour leaders such as Michael Foot and Neil Kinnock while also promising a society of entrepreneurs and mass property ownership, an attractive idea in an era of increasing materialism. Yet at the 1992 election, under Thatcher’s less secure successor, John Major, this started to change. “He didn’t want a negative campaign,” says Bale, “but he was persuaded.” When the Conservatives won the election unexpectedly, victory was attributed by many Tories – and commentators and Labour politicians – to a single crude-but-effective Conservative campaign phrase: “Labour’s Tax Bombshell.”

Further Tory election onslaughts followed: against Tony Blair’s “Demon Eyes” in 1996-7; against the possibility of a “Coalition of Chaos” under Ed Miliband in 2015; and in 2017 against the supposed “Threat to our National Security”, Jeremy Corbyn. While the anti-Miliband warnings worked well, those against Blair and Corbyn proved useless or even counterproductive. “I got embarrassed and fed up reading that Corbyn was an IRA sympathiser,” says the local Tory organiser. “Tory voters on the doorstep were beginning to feel he was being bullied, and feeling sorry for him.” Bell says: “The attacks made people feel as if they weren’t allowed to vote Labour. That’s an authoritarian position. And the attacks were so extreme, people said to themselves: ‘He can’t be that bad.’”

Facebook Twitter Pinterest A Conservative party poster from the 1997 general election. Photograph: The Conservative Party Archive/Getty Images

Occasionally during recent elections, prominent Conservatives have warned their party not to be too negative. In February 2015, May was asked at an event for Tory women whether parliamentary candidates should concentrate on positive themes or on voters’ “fear” of a Labour government. She replied: “I’m a great believer … in running a positive campaign. I think what people want to know is, what would you do for them?”

During David Cameron’s leadership, upbeat Conservative campaigning briefly returned. In the 2010 election, the slogan on the side of his cheerily multicoloured campaign bus was “Vote For Change”. But the campaign was disorganised, with no clear chain of command, and Cameron’s talk of national renewal became obscured by bleak Tory messages about the national deficit. The party failed to convert a dominant opinion poll position into a proper win.

Ever since, the Conservatives have reverted in elections to being what May once called “the nasty party”, egged on by their ever more aggressive allies-cum-overlords in the tabloids. The 2017 election saw Labour-bashing in the Sun and the Daily Mail that was even shriller and more repetitive than usual, page after predictable page on which the views of rightwing journalists and proprietors and Tory attack dogs such as Michael Fallon and Boris Johnson were indistinguishable. That the Conservatives won 42% of the vote, their highest share in a general election since 1983, and yet still did not get a Commons majority, showed the limits of this approach. Voters, far from being scared away from Labour, instead chose it in such great numbers, giving it a 40% share, its best since 2001, that the result was electoral deadlock.

“It’s a very long time since Britain lived under anything like socialism,” says Paul Goodman, editor of the influential Tory website ConservativeHome. “There probably hasn’t been a leftwing government since the 1945 [Labour] one. So it’s getting harder and harder to convince young people, especially, that socialism is a threat.” Many younger voters, not being newspaper readers, barely register all the tabloids’ anti-Labour coverage. Goodman says: “Tories have to ask quite soberly whether their campaign model is out of date.”

Facebook Twitter Pinterest More negativity: is the campaign model out of date? Photograph: The Conservative Party

They have tried to modernise it twice since the 90s. The first, ongoing effort has relied on software based on the Mosaic database system, used by businesses to identify and attract different categories of consumer. In 2004, a senior Conservative told this newspaper, “We have a new tool. It is something we have borrowed from the US”, a favoured source of campaign innovations for the Tories and Labour alike. “If you know what people’s social characteristics are, it is not too difficult to identify how they will vote.”

Such a hi-tech approach is appealing to a party that is losing its deeper connections to British society. “In 1952, the Tory party had two and three-quarter million members,” says Geoffrey Wheatcroft, author of The Strange Death of Tory England. “Now it has about 100,000.” Andrew Kennedy, a Conservative agent in Kent since 1992, says: “When I started, there were 400 professional campaign organisers across the country. Now we are down to about 60.”

Since the 90s, the Conservatives have become less attuned to social change than in their Thatcherite heyday. Britain’s expanding populations of university-educated urban liberals, recently arrived migrants and racial minorities have received, at best, occasional Tory attention during elections, and sometimes barely disguised hostility. Kennedy says this incuriosity has been a mistake: “In elections, you need to look for votes where they are not expected.” The liberal Tory journalist Iain Dale agrees: “If you want to be a national party, you need to be curious about all parts of the country.”

Instead, in 2005, the Conservatives hired Lynton Crosby. He had a formidable record in Australia of winning elections for the rightwing Liberal party. In 2013, he explained his methods at a little-noticed British charity event: “In every campaign, there are really three targets: there’s your base; then there’s the swing group; and then there’s the anti [group], who you have limited chance of ever persuading.” Crosby’s approach was “to lock your base in first”, then to concentrate on “the swing group” in marginal constituencies, and essentially to write off “the antis”.

In Australia, this narrow but canny strategy had repeatedly achieved decisive victories for the Liberals with relatively low percentages of the vote, and for almost a decade it did the same for the British Conservatives. In 2008 and 2012, Crosby was central to Boris Johnson’s election and re-election as mayor of London, focusing the Tory campaign on a “doughnut” of the capital’s more conservative outer suburbs while largely ignoring the left-leaning inner city. In 2015, Crosby oversaw a similar success in the general election, with the Conservatives’ modest 37% vote share concentrated in exactly the right places to win Cameron a majority. In 2016, Crosby was knighted for his “service to politics”.

Yet the day afterwards, Sadiq Khan was comfortably elected Labour mayor of London – despite a vicious Conservative campaign against him, guided by Crosby’s firm under the leadership of another of its founders, Mark Fullbrook. A wooden Conservative candidate, Zac Goldsmith, was simply overwhelmed by a high Labour turnout, much of it in the outer-London “doughnut”, which is rapidly becoming more racially and politically diverse. “The result was a warning,” says Goodman. “But it was not heeded.” In the 2017 general election, what Goodman calls a “flat-footed” Conservative campaign was ambushed by Labour across England and Wales.

Facebook Twitter Pinterest ‘“Strong and stable” was not resonating on the doorstep at all.’ Photograph: Bloomberg/Getty Images

Electioneering is a part of politics where one side rarely holds the advantage for long. Opponents’ innovations are copied or they are superseded. In recent British elections, the Conservatives’ clunky, old-fashioned posters – often little more than recyclings of previously successful campaign themes – and their propaganda collaborations with the tabloids have been increasingly mocked, and sometimes made to look ridiculous, by an online army of official and unofficial Labour activists.

Goodman thinks it is hard for the Conservatives, currently, to use social media with the same sort of irreverence. “The Tories were quite cheeky in the mid-70s, but they were in Opposition then. Cheekiness is something that emerges when you’re the rebels.” But that time will come, when Labour are again the ruling establishment, however hard that is to imagine now. During Tony Blair’s government, irreverent rightwing bloggers emerged, such as Guido Fawkes, and gradually made New Labour look grand and smug.

The many Tories I speak to have few quick solutions for their party’s election difficulties – which may be magnified, next time, by the need to defend 13 seats in usually hostile Scotland. The local organiser suggests that Conservative campaigning should become less centralised. “In the [2017] election, we were strongly encouraged by the party to buy into a national package of campaign material. By the end of the campaign, ‘strong and stable’ was not resonating on the doorstep at all. But it was already printed on all the pamphlets. Next time, we need to make our campaigning more grassroots, encourage constituency parties to group together, share costs, and come up with their own literature and tactics.”

In the 2015 election, the Conservatives successfully introduced another new approach: dispatching busloads of activists around the country to help capture key seats. But afterwards the tactic was investigated by the Electoral Commission, for allegedly breaching the strict legal limits on constituency spending during elections. The Conservatives were fined £70,000 by the commission, a record punishment for any party, for “significant failures” in how it had declared its 2015 electoral spending. The tactic was not repeated at this year’s election.

Facebook Twitter Pinterest Protesters outside the venue in West Yorkshire where May launched her party’s 2017 general election manifesto. Photograph: Danny Lawson/PA

In some ways, all the Conservatives’ electioneering initiatives over the past 25 years feel like short-term fixes, for a party that has still not worked out how to present itself for the long term, given the spread of social liberalism, and the gradual seeping away of public confidence in the Thatcherite dream of the free market, a process greatly accelerated by the 2008 financial crisis. Goodman says: “The Conservatives haven’t tried to educate the public about their core beliefs for a long time, probably not since Thatcher’s time. And the landscape has changed for capitalism since then. Capitalism isn’t producing the well-paid blue-collar and white-collar jobs in the numbers it was.” Or as Dale puts it: “People think there is something deeply wrong with the economy, and they think it is something to do with the Conservatives.”

Labour’s own electioneering challenges – how to hold together and further widen its broad coalition of supporters; how to win next time as possible favourites, rather than exceed low expectations as underdogs; and how to keep its hordes of optimistic activists energised if it doesn’t win – mean that the Conservatives are likely to have plenty of opportunities in general elections to come. But if they are ever to recover the electoral dominance they enjoyed under Thatcher, and for long stretches of British political history before that, better campaigning may not be enough. Goodman says: “We need something big in policy … something to shift the balance between rich and poor, or between the north and south. We need to get our next manifesto right before we worry about the machine to deliver it.” Gallingly for Conservatives, that sounds rather like how Corbyn’s Labour party does things.