The Wittgensteins, ensconced in their grand Winter Palace in fin-de-siècle Vienna, were hardly a model family. The father, Karl , was a brutal autocrat as well as a high-class crook. He was an engineer by vocation, and his son Ludwig would later do some original work in aeronautics at Manchester University. A fabulously wealthy steel magnate, Karl rigged prices, bleeding his workers dry and doing much the same to his timorous wife Leopoldine. She once lay awake all night, agonised by an ugly wound in her foot but terrified of moving an inch in case she disturbed her irascible husband. She was an emotionally frigid mother and a neurotically dutiful wife, from whom all traces of individual personality had been violently erased.

The family was a seething cauldron of psychosomatic disorders. Leopoldine was afflicted by terrible leg pains and eventually went blind. Her children had their problems too. Helene was plagued by stomach cramps; Gretl was beset by heart palpitations and sought advice from Sigmund Freud about her sexual frigidity; Hermine and Jerome both had dodgy fingers; Paul suffered from bouts of madness; and little Ludwig was scarcely the most well balanced of souls. Almost all the males of the family were seized from time to time by bouts of uncontrollable fury that bordered on insanity.

Behind Karl the prosperous bourgeois lay a madder, more reckless man. He ran away from home at 17, boarded a ship bound for New York and joined a minstrel band. Before making his pile in Vienna he was a restaurant violinist, a night watchman, a steersman on a canal boat, and taught the tenor horn in an orphanage. Despite being one of the premier families of the Austro-Hungarian empire, most of the Wittgensteins were spiritual outlaws and adventurers. They combined the aristocrat's cavalier disdain for convention with the underdog's suspicion of authority.

The sons of the household had a distressing habit of doing away with them selves. Handsome, intelligent, homosexual Rudolf strolled into a Berlin bar, dissolved potassium cyanide into his glass of milk and died in agony on the spot. Two years earlier, Hans Karl had disappeared without trace and is thought to have killed himself at sea. He was a shy, ungainly, possibly autistic child with a prodigious gift for maths and music, whose first spoken word was "Oedipus". He, too, was thought to be gay. Kurt seems to have shot himself "without visible reason" while serving as a soldier in the first world war. The philosopher Ludwig claims to have begun thinking about suicide when he was 10 or 11.

Paul, a classmate of Adolf Hitler, became an outstanding concert pianist. Unusually for male members of the family, he was robustly heterosexual. The Wittgenstein ménage was more like a conservatoire than a family home: Brahms, Mahler and Richard Strauss dropped in regularly, while Ravel wrote his "Concerto for the Left Hand" specially for Paul, who had lost an arm in the first world war. Paul thought his brother Ludwig's philosophy was "trash", while Ludwig took a dim view of Paul's musical abilities. The Winter Palace resounded with constant yelling and vicious squabbling.

Entrusted with the family fortune after his father's death, Paul invested it unwisely in government war bonds and lost most of it. Ludwig still inherited a sizeable amount of money, but in Tolstoyan spirit gave it all away to three of his siblings. His rooms in Trinity College, Cambridge, were almost bare of furniture. He is said to have remarked that he didn't mind at all what he ate, as long as it was always the same thing. It was a far cry from the overbred Vienna of his youth. Ludwig's monkish austerity, evident in the style of his first great work, the Tractatus Logico-Philosophicus, was among other things a reaction against a Viennese world of cream cakes and swollen bodies, in which many of the poor slept in caves or parks.

Unconsciously re-enacting his father's impulse to escape, Ludwig fled from Cambridge to become an assistant gardener in an Austrian monastery, sleeping in a potting shed. He also lived for a while in a remote cottage in the west of Ireland, shacked up on the edge of a Norwegian fjord, and taught as a schoolmaster in several Austrian villages. Perhaps all this was a spiritual version of his brothers' suicides, on the part of a man seized by spiritual torment and self-loathing. If he inherited his father's instinct to scarper, however, he was also lumbered with his crazed bouts of fury. In one village school, he hit a girl so hard that she bled behind the ears, and then belaboured a boy about the head until he slumped unconscious to the floor. While Ludwig was dragging the boy's body off to the headmaster, he bumped into the irate father of the girl whose ears had bled, dropped the unconscious boy and did a runner.

Alexander Waugh's eminently readable, meticulously researched account of the Wittgenstein madhouse might have speculated a bit more on how this background helped to shape the most celebrated of all the Wittgensteins. It certainly casts some light on Ludwig's extraordinary contradictions. Haughty, imperious and impossibly exacting, driven by a fatiguing zeal for moral perfection and contemptuous of most of those around him, he was a true son of patrician Vienna. Yet his greatest work, Philosophical Investigations, also represents a rejection of this world in its embrace of the ordinary, its acceptance of the imperfect and incorrigibly plural.

Wittgenstein was an arresting combination of monk, mystic and mechanic. He was a high European intellectual who yearned for a Tolstoyan holiness and simplicity of life, a philosophical giant with scant respect for philosophy. He could never really decide whether he was a Brahmin or an "untouchable". Much of this makes sense if one sees it as an ambivalent relationship to his family background.

On the one hand, he tried to divest himself of all that pomp and excess. If he was sometimes plunged into spiritual despair, it was because he was unable to strip himself of himself. Wittgenstein struggled to live on what he called the rough ground of everyday life.

As a man who hailed from an Austro-Hungarian empire inhabited by Germans, Slovaks, Romanians, Serbs, Slovenes, Magyars and a good many other quarrelsome ethnic groups, he came to see human cultures as inherently diverse. But he was also haunted by a lofty, lethal vision of purity (what he called the pure ice), which was a product of his background and a form of rebellion against it. And the fact that he was torn between the rough ground and the pure ice was the source of much of his sorrow. Perhaps his brother Hans's first word sums it all up: "Oedipus".

• Terry Eagleton wrote the screenplay for Derek Jarman's Wittgenstein. To order The House of Wittgenstein for £18 with free UK p&p call Guardian

book service on 0870 836 0875.

· This article was amended on Saturday November 22 2008. The above review mentioned some of the children of Karl and Leopoldine Wittgenstein and suggested that Hans, Hermine, Gretl, Jerome, Paul, Rudolf and Ludwig were siblings. In fact Jerome (Stonborough) was Gretl's husband, not her brother.