The first volume of Simon Schama’s mammoth undertaking, The Story of the Jews, ended two and a half thousand years after it began, with the Spanish Inquisition and the expulsion of Jews from Iberia. The second volume, entitled, with more than a smidgen of irony, Belonging, begins in the Venice ghetto, where many victims of that expulsion found uneasy refuge.

Some had fled from Portugal, where, during the Easter of 1506, about 2,000 “New Christians” (Jews who had been forced to convert) were slaughtered in three days. “The ostensible cause,” writes Schama, “was a vocal comment made by a New Christian in church to the effect that a miraculous illumination on the face of the Saviour on the cross might have been a mere effect of candlelight.”

That incident warrants only passing mention, but it’s a shocking reminder of just how vulnerable Jews have been in Europe over many centuries.

Belonging, which covers the period from 1492 to 1900, is concerned with the Jewish search for security and the efforts – both coerced and voluntary – at assimilation in Europe (there are brief excursions to other settlements, in America, and as far off as China).

Jews have traditionally been caught in a double bind: not trusted as a distinct minority, and trusted even less when they attempt to adopt the majority culture/religion.

It’s a position that has led to repeated cycles of persecution, expulsion, confinement and a ceaseless hankering to be accepted. That, in essence, is the story of the Jews and Schama lays it out in rich, complex and fascinating detail.

Although this is an ambitious doorstop of a book, Schama is not interested in history writ large. His signature method is to recount the plight of individuals against the swirling backdrop of events. It’s a high-wire approach that can leave the reader wondering if the extended anecdotes – a tragic conman in 16th-century Venice, rumours of a Jewish sex cult in 18th-century Moldova – will ever reach the firm ground of historical import.

Mendoza had abundant charm and promotional savvy. These, plus some fearless performances, would make him a national hero

Yet even in the midst of what might at first appear elaborate digressions, Schama maintains the attention with the vividness of his writing and his talent for unearthing gripping figures full of human contradictions.

And it’s through this dazzling immersion in the preoccupations of the period that the bigger picture slowly emerges. History, you learn, is what happens to you when you’re busy trying to survive.

The bigger picture in 16th- and 17th-century Europe is of a continent driven by trade but riven by religious disputes and wars. The place of Jews, at least those who manage to thrive, is both critical and precarious. They are the eternal outsiders whose only option is to make themselves indispensable in those areas – trade and finance – in which they’re allowed to participate. But in turn any success they achieve is held against them.

Even if we are all too familiar with the concept, the protean spectre of antisemitism that haunts these pages is sobering. And it’s impossible to read of the countless precedents without reflecting on modern-day variants that continue to afflict political discourse.

The hatred that manifests in the euphemism “anti-Zionist” serves today to normalise the ever-embattled condition of Jews who, by dint of being Jewish, are viewed as part of an international conspiracy to deny Palestinian rights. If that seems hyperbolic, why is it a typical sight in Britain and Europe to see Jewish schools, community centres and synagogues under guard – armed guard in France?

The answer to that question cannot – or should not – simply be framed by the disputed territory of Palestine. It used to be, as Schama notes, that the Jews were not trusted – were seen as rootless and disloyal – because they didn’t have a state of their own. It’s more than ironic that the reason they now draw suspicion is because there is a Jewish state.

The causes of the exclusion and persecution of Jews vary across time and territory. But the given reasons were often pretexts. Jews, for example, were frequently viewed in France as un-French because they had little agricultural relationship with the land. But in Poland and Russia, the Jewish peasant was a commonplace – as were pogroms.

So why have the Jews for so long occupied this dangerous terrain between social exclusion and economic toleration? Religion, specifically the teachings of those expansionist offshoots of Judaism – Christianity and Islam – certainly hasn’t helped. But there is also another psychology at work, which is to do with how a minority is perceived when it refuses to be limited by the limitations imposed upon it.

That is a recurring theme of the book, the struggle to transcend the social stigma and professional prohibitions of being a Jew. It’s a narrative that could easily be rendered as a stirring tale of noble victims overcoming mindless victimisation.

But Schama is too subtle a writer and historian to succumb to that temptation. He is not afraid to examine Jewish corruption and double-dealing, the kinds of misdeeds that attend all forms of power and influence, but that have been the subjects of so much myth and misrepresentation in the case of the Jews.

In among a colourful gallery of innovators, showmen and rogues, there are, though, some wonderfully inspiring stories. My particular favourite is that of Daniel Mendoza, one of the great bare-knuckle boxers of Georgian England. Schama paints a compelling portrait of aspiration among poor London Jews and the East End slums that Mendoza literally fought his way out of. This barrel-chested son of Sephardic Jews had, according to one contemporary boxing writer, “a bottom never impeached, and possessing wind that was seldom disordered”.

As well as rectal fortitude and staying power, Mendoza was blessed with abundant helpings of charm and promotional savvy. The combination, plus some fearless performances, would make him a national hero.

It’s perhaps not a coincidence that Mendoza’s hugely popular pugilistic progress prefigured the Jewish emancipation that would eventually lead in the mid-19th century to admission to parliament. “[S]omething had happened between Christian Britain and the Jews,” writes Schama, “something unlike a connection made anywhere else, even in liberally tolerant America. What that something eventually turned out to be would change the history of the world.”

This profoundly illuminating book ends at the close of the 19th century with the scandalous antisemitism of the Dreyfus affair and a dejected Theodor Herzl turning to Jerusalem. After 3,000 years of struggle, even Herzl could not have imagined that the most harrowing and eventful century in Jewish history was yet to come.

• The Story of the Jews, Volume Two: Belonging, 1492-1900 by Simon Schama is published by Bodley Head (£25)