Think of it as Israel’s groundhog night. It begins with exit polls that delight Benjamin Netanyahu’s opponents and send a shudder through Likud headquarters, and ends several hours later with a crowd of supporters cheering “Bibi, king of Israel”. It happened that way in 1996, when Netanyahu won his first term as prime minister, it happened again in 2015, and it happened once more last night. The initial projection, which saw Netanyahu lagging behind Benny Gantz, a former chief of staff of the military, melted away as the night wore on, until both men had an even number of seats – but with Netanyahu the obvious winner thanks to the overall strength of the bloc of rightist and religious parties that he calls his “natural partners” in coalition. Not all the votes have been counted, but barring an arithmetical miracle he will soon embark on his fifth term in power.

How did he do it? Credit belongs in part to his own mastery of politics’ dark arts. At Likud HQ, his supporters hailed him as a “magician”, and it is true that there is no more able sorcerer when it comes to playing on the prejudices and fears of his base. (Donald Trump is a novice by comparison.) I was in Israel for the last few days of his campaign, watching as Netanyahu pressed the panic button in a bid to bring rightwing voters back home to Likud rather than opting for any of the smaller, even more ultra-nationalist parties. He went on the radio, warning that he was “losing” the election, persuading his base that the only way to keep out Gantz – whom he repeatedly branded as “left” and “weak” – was to give Likud the status of largest single party.

But it was uglier than that. Four years ago, Netanyahu rallied his troops by warning that Arab voters were heading to the polls “in droves”. Yesterday he played a similar trick, with a video warning that his opponents “were plotting with the Arab parties to form a government”. The message was clear in both cases: one fifth of Israel’s citizens are not legitimate participants in the democratic process, and their inclusion is to be feared. Once again, it worked. The whole effort was reinforced by a tactic so base it should beggar belief – except it’s all too believable. Likud placed 1,200 hidden cameras in Arab polling stations, for what the party claimed was an effort at election monitoring. The cameras were found, word got out and the dirty trick achieved its objective – deterring Palestinian citizens of Israel from exercising their democratic rights. (In parts of that community, voting is regarded as collaboration with the Israeli state, so the risk of being seen to vote could be a powerful disincentive.) That tactic too seemed to work: Arab turnout was down significantly, reduced to single digits in some Arab towns and villages.

As for the wider population, among the Israeli Jewish majority Netanyahu’s success is not that hard to explain. As Aluf Benn, editor of the liberal Ha’aretz newspaper, put it to me, “Bibi” has provided “identity, stability, security and (relative) prosperity”. Security might be the most important of those, in both the economic and national senses of that word.

He can boast of a relationship with the American president that has delivered US recognition of not only Jerusalem as Israel’s capital but, in the last fortnight, Israel’s annexation of the Golan Heights. Less than a week before polling day, Netanyahu was in Moscow to see his pal Vladimir Putin, who gave him another pre-election gift to take home: news that Russian troops in Syria had found the remains of an Israeli soldier killed in 1982. Netanyahu has welcomed to Israel Narendra Modi of India and Jair Bolsonaro of Brazil, as well as enjoying warm relations with China and the Visegrad bloc of Poland, Hungary, the Czech Republic and Slovakia. Netanyahu may not win any popularity contests in the liberal capitals of north-western Europe, but he can argue that it is the likes of Paris and Berlin that are isolated, while it is the “illiberal international”, the world of Trump, Putin, Xi Jinping and Viktor Orbán, that represents the future – and which embraces him as a valued ally.

The way many Israelis see it, the conflict with the Palestinians is not solved – and may indeed be insoluble – but, under Netanyahu, it has been reduced to a manageably low level. The shootings of protesters at the Gaza frontier continue, but they rarely make headline news. Instead, most Israelis think that over the last decade their prime minister has brought them relative peace and quiet. Sure, they might say, there was the bloodshed of 2014’s summer offensive in Gaza, but that episode is the exception rather than the norm. There has been no repeat of, for example, the Lebanon war of 2006, launched by Netanyahu’s predecessor, Ehud Olmert. In this view, he talks loudly, but rarely wields the big stick. (Intriguingly, I heard the arguments of some on the Israeli far left who were nervous at the prospect of a Gantz victory, fearing the former commander might be less “militarily restrained” than Netanyahu has proved.)

Couple that with the relative prosperity of Israel, with the economy ticking along nicely, and you have a clue as to why re-electing Netanyahu seemed, to many, like the stable, safe option. The major corruption inquiries against him were not disqualifying, partly because he has been effective in persuading his base to see those the way Trump’s base saw the Mueller investigation: as an establishment witch-hunt against him. It is part of Netanyahu’s sorcery that this son of the Israeli elite, set to be in power for longer than any PM in Israeli history, still manages to project himself as the plucky outsider, tribune of those Israelis who’ve long felt marginalised and disdained.

All this helps explain why Netanyahu was able to win more seats than ever. But what will allow him to remain prime minister is the overall success of the broader right, which will form his coalition. It dominates in Israel because the left has been truly trounced, its one-time core message – that Israel should offer land in return for peace with the Palestinians – thoroughly discredited in the eyes of Israelis, ever since the failure of peace talks at Camp David in 2000 and the intifada that followed, as well as the 2005 withdrawal from Gaza that brought not peace but conflict and the rise of Hamas. It’s this structural defeat which explains why even a clutch of former generals could not beat Netanyahu: Israel has shifted to the right and to be on the other side, to be branded “left”, is fatal. (Although, given that context, there’s some comfort in the fact that nearly half the population did back anti-Netanyahu parties – including the small constituency for those, such as Labour and Meretz, who still campaign for a two-state solution to the Palestinian-Israeli conflict.)

What will Netanyahu do with this new mandate? Benn is surely right to predict that the coalition will see a trade-off – “immunity in exchange for sovereignty”. Netanyahu’s coalition partners will rewrite the law to protect the PM, and in return he will move ahead with annexing parts of the occupied West Bank – a promise he had long refrained from making, until the campaign’s final weekend. There was a time when no Israeli PM would have dared make such a move, fearing the US response. But Trump offers no such restraint.

So Palestinians will have to brace themselves for a Trump “peace plan” that is likely to deny them the territory they need to build a state of their own. Meanwhile, Netanyahu’s victory promises a further assault on democratic norms and the rule of law inside Israel. It surely spells gloom for the long-term prospects of both peoples, but they are used to that by now. It’s been this way on and off for most of the last quarter century. For truly this is the age of Netanyahu.

• Jonathan Freedland is a Guardian columnist