To those of us who hoped that Barack Obama’s election marked a departure from right-wing rule, the president’s failure of leadership has been stunning. Seldom have insurgent expectations – even sceptical, guarded ones – been deflated so swiftly. From the moment he announced his staff and cabinet appointments (Rahm Emanuel, Timothy Geithner, Lawrence Summers, Hillary Clinton, Robert Gates et al) it was clear that Obama meant to play by the same Washington rules that created the policy disasters he inherited from George W. Bush. Obama had retreated into politics as usual. He never looked back. One did not have to be a sentimental utopian to be disappointed.

In domestic affairs, Obama’s obeisance to the Washington consensus led him to abandon the bold approach he articulated during the campaign in his Philadelphia ‘speech on race’, when he attacked the manipulation of racial hostilities to divide the black and white working class. No one running for high office in America had done that since the 1890s. But once in power, Obama soon abandoned any pretence of promoting social democracy. After pushing through a stimulus package, he quickly (and illogically) embraced the gospel of austerity preached by Tea Party ranters and ‘centrist’ pundits. Indeed, the president played a critical role in legitimating this corrosive creed, which despite its tendency to exacerbate recession has now become orthodoxy on both sides of the Atlantic. In his first State of the Union address, given in January 2010 when his party still had control of both houses of Congress, Obama announced a three-year freeze on non-defence discretionary spending – a move he had called an ‘example of unfair burden sharing’ and ‘using a hatchet when you need a scalpel’ when John McCain proposed it during the campaign of 2008.

In the same speech, Obama embraced the false analogy between federal budgets and household budgets, overlooking (for starters) the government’s control of taxation and the money supply. ‘Families across the country are tightening their belts and making tough decisions,’ he said. ‘The federal government should do the same.’ His puzzling timidity culminated in his supine response to ‘deficit hawks’ during the debt ceiling crisis of the summer. Recently, Obama’s spine has begun to stiffen in response to the Occupy movement. He has finally started to make a populist critique of systemic inequality, but it remains to be seen how sustained or effective this new stance will be.

Obama’s deference to established power has been even more striking in national security affairs. Yet this was the area where he had promised the most. Having opposed the Iraq War ‘not just in execution but in conception’, he seemed to harbour a healthy scepticism toward interventionism. And he assured civil libertarians that he would reverse the Bush administration’s drive towards executive tyranny. He would renounce torture and the ‘extraordinary rendition’ of prisoners to CIA ‘black sites’ where torture took place. He would close Guantánamo, and end indefinite detention and warrantless surveillance. So we were led to believe.

But in practice President Obama has been far more committed to continuity in national security policy than candidate Obama promised. Despite fitful rhetorical displays of dislike for torture, the Obama administration has continued extraordinary rendition, indefinite detention and warrantless surveillance, while expanding the doctrine of state secrets used to conceal those practices from public view. Guantánamo is still in business, beyond the reach of civil law. Though the phrase ‘war on terror’ has fallen into official disuse, the carte blanche it provided for foreign military interventions remains intact. Providentialist assumptions still lend US imperial adventures an aura of sanctity. Obama’s Nobel Prize acceptance speech dragged out all the old tropes, including a familiar set-piece: the United States on the frontiers of freedom, fighting a 60-year war against tyranny that culminated in the battle for Afghanistan. Apart from the echoes of Reinhold Niebuhr – the furrowed brow, the feigned reluctance to use force – the words could have been spoken by George W. Bush.

Among his apologists, Obama’s capitulation to convention has two main explanations. One emphasises the regimented hostility of his opponents, fuelled by racially charged resentment of a black man in the White House. No one can blame Obama for his reluctance to provoke racist violence, yet no matter how cautiously he proceeds, he can never satisfy his right-wing detractors. The other explanation is that Obama was never that radical in the first place: his approach to politics, shaped by his experience as a community organiser in Chicago, has always been conciliatory. He really believes in the banalities of bipartisanship. But this fails to account for his hasty retreat from boldly articulated campaign positions, which suggests something more complex than a politician backing away from promises he knew he could not keep.

Focusing the picture requires us to ask questions about the origins of the president’s temperament. How did Obama develop his paradoxical combination of fierce ambition and persistent timidity? Part of the story lies in his chequered family background. Two recent books, biographies of his father and his mother, may help to provide a genealogy of disappointment – a fuller explanation of how Obama came so grievously to disappoint his supporters as well as, perhaps, himself.

Sally Jacobs’s The Other Barack tells the sad story of the president’s father. Despite occasionally pedestrian prose, the biography is unsparing in its detail and unsentimental in its judgments. Barack Obama Sr emerges as a minor tragic figure. He was smart, ambitious and eager to join the first generation of Kenyans to govern their country after the end of British rule. He was also afflicted by overweening arrogance, dogmatic certainty and a self-destructive fondness for Johnnie Walker Black. In spite of his extraordinary talent, which led him from the back-country of Luoland in western Kenya to the graduate economics programme at Harvard, he ended his life at 46 as a mid-level bureaucrat in Nairobi, a hopeless drunk, recently married and a new father but estranged from nearly all his previous wives and children. Driving home from a bar after a long night of ‘double-doubles’, he crashed into a eucalyptus stump and was killed instantly.

Barack Sr’s pathologies had their own genealogy. His father, Onyango Obama, was a house servant in Nairobi, where he became obsessed with English ways. A harsh patriarch, he terrorised his wives and children, and intimidated most of the people in the village with his hyper-organised household. Despite his contempt for rural Kenyans’ backwardness, he derided British assumptions of superiority. Christianity, he was convinced, fostered submission to imperial rule: ‘Only a fool would show mercy towards an enemy.’ Converting to Islam, he became Hussein Onyango in his mid-twenties. Frustrated by his first wife’s inability to have children, he abducted a second, Akumu Njoga, who became Habiba with her own conversion. Baraka (‘blessing’ in Arabic) was born in 1936.

Baraka was petted and adored even by his father, who ignored the boy’s sisters. He became Barack at six, when he converted to Christianity. (His father, not a particularly observant Muslim, was apparently untroubled by the conversion.) Barack went to a mission school run by Seventh-Day Adventists, walking three miles but, unlike his classmates, wearing shoes. His father told him: ‘I want you to go beyond where I am.’ On returning home each day, the boy was required to recite sums to his father while standing by the dinner table. If he was slightly less than perfect, he was forbidden to sit down and eat or locked in his room for the night. The family owned a gramophone, but strict rules governed its use. ‘Barack always wanted to dance,’ his friend Akello recalled, ‘but Onyango only let him have so much. A couple of songs. One record. Then it was over.’ Pleasure had to be carefully rationed.

When Habiba withdrew from him, Onyango turned to other women, including the one who became ‘Mama Sarah’, a surrogate grandmother to the president. He also took to beating Habiba. One night, after he nearly killed her, she fled to her family in Kolonde. Barack and his sister were heartbroken. They walked 100 miles to try to get her back. She refused. The children’s father came to retrieve them, citing the proverb he would use to describe his son’s wandering ways: ‘For the bird the world is never too far.’

At school, Barack developed the insistent argumentative style he deployed throughout his life: ‘You don’t know what you are talking about. I’m telling you what I know.’ He was a disobedient student, but sailed through the Common Entrance examination for entry to Maseno, an elite Anglican school founded in 1906 for the sons of Luo chiefs, which became the training ground for the first wave of Kenyan nationalists. Barack broke the rules all the time – sneaking off campus to drink with village rowdies, walking on the sacrosanct school lawn. Eventually he was expelled. He was not allowed to go forward for the Cambridge School Certificate, the ticket to further education. When Onyango heard the news, he beat Barack bloody.

Amid the Mau Mau rebellion and the rise of nationalists like Jomo Kenyatta, Barack headed for Nairobi. He accepted a clerkship with an Indian law firm and fell in with the trade-union movement, including the charismatic Tom Mboya, who would become Kenyatta’s chief lieutenant. Mboya, like Barack, was arrogant and self-assured, an eligible young bachelor who liked to put on a tuxedo and dance with elegant young women. Barack was a champion dancer, Jacobs says, ‘his lithe form bumping out a rhumba or twirling his partner in the fast-paced mach dance, his ever present Sportsman cigarette angled rakishly from his full lips.’ He met his first wife, Grace Kezia Nyandega, on the dancefloor. They were married in 1957 under Luo customary law, by communal consent.

Barack was impatient with his job and thought the work beneath him. Mathematically sharp, he yearned to be an economist. Among the activists in Nairobi was Sara Elizabeth Mooney, a 43-year-old Texan and literacy teacher, who offered Barack a job as her secretary. He cultivated an academic look, with horn-rimmed glasses and pipe. Mooney found him exceptionally promising: his ebullience, his self-confidence and his ‘trumpeting voice, now matured, that could snap a sleepy room to attention from a corridor away’. She coached him in the process of applying to American universities. When Barack was admitted to the University of Hawaii, his triumph was celebrated in the Luo villages of his childhood. He was going to be a Big Man. But his wife Kezia was not so happy, and even Onyango was ‘oddly heartbroken’. The bird was taking flight again. Still, he needed money. He raised some from the African American Institute and the African American Students Foundation, but most of it came from Mooney.

In Hawaii, Barack tried his best to reinvent himself. He never mentioned his wife and children back in Luoland unless it served his interests. He carried a briefcase, wore button-down shirts, black lace-up shoes, sometimes even a tie. School was serious business. Invited to discuss a proposed East-West Center with the university president at a posh cocktail party, he glimpsed the world he wanted to inhabit. For relaxation, he hung out with the aspiring intellectuals who constituted a would-be counterculture. One of this group, Neil Abercrombie (later a US Congressman and governor of Hawaii), recalled Barack discoursing on colonialism and its aftermath, but also that he had no patience with those he deemed inferior, which was almost everyone; he talked past them, jabbing his pipe.

Not everything was earnest argument. The self-conscious outsiders ate pizza; they listened to Sonny Terry and Brownie McGhee singing the blues; they talked Kafka, Dos Passos and Zen masters. There was a lot of sophomoric posturing but also a widespread feeling, however vague, that a dramatic social transformation was in the air. The grandiloquent rhetoric of John F. Kennedy fed the recurrent American dream of world regeneration. As a black African on the cutting edge of his country’s drive for independence, Barack was determined to be at the centre of whatever shifts occurred.

The East-West Center encouraged global ambitions and enticed the hopeful young. One of them was a girl from Seattle called Ann Dunham, who became Barack’s wife and the president’s mother. Her father Stanley was restless and ineffectual, a furniture salesman full of big plans and bad jokes; her mother Madelyn was ambitious and competent, a bank vice president whose favourite colour, according to her daughter, was beige. The couple started out in Wichita, Kansas, where Stanley Ann was born in 1942. (The name was a homage to her father but also to Bette Davis, Madelyn’s favourite actress, who played a character called Stanley in a movie released that year, In This Our Life.) Always looking for the next big break, her father kept the family on the move, bumping around the American West for years until they finally came to rest for a while on Mercer Island, Washington, near Seattle.

Stanley Ann was self-sufficient and wilful. Her sympathetic biographer, Janny Scott, pictures her dreaming of escape from provincial boredom as she pored over stacks of National Geographic magazines. She hated the jock culture of her high school and took refuge in the proto-bohemian set of wise-ass boys who drank espresso, watched Satyajit Ray films, and listened to recordings by Carlos Montoya and Dexter Gordon. She befriended closeted gays and other misfits, sharing their highbrow pretensions and their sense of marginality. Soon after she graduated, her father smelled another business opportunity and moved the family to Honolulu, where she dropped the name Stanley and enrolled in the University of Hawaii in September 1960.

Russian class was the perfect place for a girl who liked foreign movies and jazz to challenge Cold War pieties. It was also a good place to catch the eye of a handsome black stranger. She and Barack arranged to meet after class, on a bench outside the university library. He was an hour late, and she fell asleep. When he arrived, Scott writes, ‘the man from Kenya awakened the girl from Kansas, literally and figuratively.’ This mythic scene, which their son recounted in Dreams from My Father, supposedly embodied Ann Dunham’s approach to life, as a friend described it: ‘to just be herself in the world … just leave yourself open to the world when you’re sleeping.’ Her awakener was a big shot at the East-West Center, famed for his fearless intellect and heavy drinking. He frequently passed out at parties, where departing guests would have to pick their way around him. But Ann knew nothing of this, at first. She was smitten by his looks, his intelligence and his ‘seductive, almost hypnotic voice’. It was, one of his classmates said, ‘the most mellow, deep voice’ he had ever heard, ‘with a slightly African articulation and maybe a flavour of Oxford’. He was going to be a Big Man in the new Kenya – not ‘the sort of man who would have carried a condom in his wallet’, as one of Ann’s graduate school friends later observed. What was a free-spirited American girl to do?

Their courtship was swift, intense and one-sided. As Abercrombie observed, Barack ‘was much more in love with his intellect than with a woman’. By early November, about the time Kennedy was squeaking out a victory over Richard Nixon, Ann became pregnant. She was still a few weeks shy of her 18th birthday. They were secretly married on 2 February 1961 in Maui, but it is unclear how long they actually lived together. Scott reports that Ann’s friends suspected she knew early on ‘she was in over her head with this guy.’ If the food she served displeased him, he would hurl his plate against the wall. Life with an African patriarch began to look less appealing. Even before Barack Jr was born, on 4 August 1961, Ann had already moved back in with her parents. They were ‘somewhat awed’ by her husband on the few occasions they met him and did their best to suppress any distress they felt about the marriage and separation. Onyango Obama, by contrast, wrote to Stanley Dunham that he ‘didn’t want the Obama blood sullied by a white woman’, and spurned his son’s new family.

He needn’t have worried. As Jacobs makes clear, Barack Sr was more concerned with succeeding in Kenya than with keeping his American family together. In 1961 he applied for admission to graduate economics programmes in the United States, concealing his white wife and baby, mentioning only Kezia and their children in Africa. He was offered a generous package at the New School in New York but only tuition at Harvard. There was no question which choice he would make. Penury at Harvard provided a convenient excuse for leaving his new wife and child behind, as well as a first-class seat on the express train to international prominence. As Dunham later told her son: ‘Barack was such a stubborn bastard, he had to go to Harvard. “How can I refuse the best education?” he told me … That’s all he could think about, proving that he was the best.’ Musing on the similarities between father and son, Jacobs imagines them meeting on the street in Cambridge, two ambitious outsiders, each recognising ‘strokes of himself in the other’. For both, she writes, ‘their Harvard pedigree would ultimately become an aspect central to their identity, though in sharply contrasting ways.’ For the son, it was the springboard to multicultural celebrity; for the father, the failure he later tried to pretend was success.

Harvard in the early 1960s was pervaded by the Kennedy mystique and the mantras of ‘development economics’. That was Barack’s area of interest, at a moment when the economics department, like the discipline in general, was shifting from macroeconomics, with its focus on institutional and historical context, to econometrics, with its reliance on abstract mathematical models. Barack was a crack mathematician and held his own among his formidable classmates, who included Lester Thurow and Samuel Bowles. Remaining aloof from the white mainstream, he hung out with the African students. ‘He spoke very firmly to us about education and what we needed to do,’ said one. ‘He sounded just like President Obama does now.’ But he also spent a lot of time drinking and chasing women. Dunham learned about his escapades and filed for divorce early in 1964. The problem, for her, was not so much the girls as the money he spent on them rather than on his son. Barack’s plans unravelled soon after. The head of Harvard’s International Office learned from the Immigration and Naturalisation Service that Obama had two wives, and refused him any further support in Cambridge. Barack was told to go to Kenya to write his thesis. The INS turned down his request for an extension of his stay. His soaring ambition had stalled.

Back in Kenya, his Harvard experience became part of his Big Man persona. He referred often to his Harvard training and insisted on being called ‘Doctor’ even without the degree. It was a great time for young black professionals in a newly independent African nation. A management trainee for Shell/BP, Barack stood out in his bespoke suits, silk ties and shiny shoes, roaring around Nairobi in a blue Ford Fairlane with racing stripes. Beneath the bravado was a desperate desire for recognition.

That became apparent when Ruth Baker arrived. She was a Simmons College graduate who had fallen in love with him in Cambridge, and whom he had promised to marry if she joined him in Kenya. When she showed up, he asked his friend Samuel Ayodo, the minister of natural resources, to ‘tell Ruth that my father is a king and my family is very, very important.’ Barack was painfully eager to impress Ruth, and to impress everyone else with his white wife. He had no intention of living ‘like an African’, he told his friends, with more than one wife in his household at one time. Kezia would not be included.

Ruth soon learned that African men stayed out drinking and partying every night. But her husband started earlier in the day and was more relentless than most of his colleagues. Barack would stumble in late, bellowing at his wife for being asleep, beating her for failing to fix him food, scandalising the neighbours, terrifying the children – who included two sons by Ruth and a son and daughter by Kezia. (He brought Kezia’s children into his family but refused to let their mother visit them.)

Barack’s decline fell into a pattern. He would acquire a decent job based on his econometric skills – with the Central Bank of Kenya, then the Kenya Tourist Development Corporation – but would soon begin complaining: ‘The money was never enough,’ Jacobs writes, ‘he was employed below his abilities,’ or those above him were out of their depth. When he was in his cups, which was most of the time, he would denigrate his superiors’ credentials and claim more status than he actually possessed.

Some of his difficulties could be attributed to circumstances beyond his control. The Kenyatta government was dominated by Kikuyus; Barack was a Luo. He could never suppress his tendency to bluntness. His criticisms of Kenyatta’s development plans were often on the mark. He anticipated the catastrophes created by unregulated capital and proposed instead the melding of free market and communalism in land co-operatives. But accurate criticism was not the path to power. Barack found he could not play the fool to curry favour with the Europeans at the Central Bank of Kenya. He lasted nine months. After Mboya, also a Luo, was assassinated, Barack publicly accused the Kikuyus, denouncing Kenyatta’s ‘betrayal of the Kenyan people’. This was not the sort of conduct that would encourage the Kenya Tourist Development Corporation to overlook his alcoholic over-reaching. He was fired, drank himself insensate and drove his car into a tree, breaking his leg in several places. Ruth became the family’s sole support.

No longer a Big Man, but still trying to look the part, Barack flew to Hawaii in 1971 for a reunion with his son. The father seemed fragile and walked with a limp, but still held forth with his enveloping, authoritative baritone voice, trying fitfully to assert his authority over a son he had not seen since he was a baby. And for a couple of weeks he pulled it off. In Dreams from My Father, Barack Jr remembered the power of his father’s voice to compel attention and respect, to summon the whole family back to the hopeful atmosphere of the early 1960s. But one night when the boy flipped on the television to watch How the Grinch Stole Christmas, his father told him to turn it off and do his homework: ‘Barry, you do not work as hard as you should. Go now, before I get angry at you.’ Barry retreated to his room, counting the days till his father left. Barack Sr stayed long enough to put in a triumphant appearance at Barry’s school, an exotic, articulate representative of the New Africa. He preserved that mythic stature in his son’s imagination until, as a young man, Barry learned from his half-sister Auma about their father’s many failings and pathetic dénouement. After the Honolulu visit, the elder Obama returned to Nairobi to continue drinking himself to death. Late one November night in 1982, he finished the job. Dreams from My Father is in part the son’s attempt to come to terms with the fallen idol.

Obama’s mother receives little attention in that book, though she was the parent who actually raised him. Scott’s A Singular Woman fills in the blanks, reconstructing the peripatetic life of Ann Dunham after she separated from her son’s father. Scott is at pains to redeem Dunham from the hippy-dippy aura that surrounds her in Dreams from My Father. She was ‘tough, sharp and worldly’, Scott insists, even if she did wear rubber sandals and batik tent-dresses. The redemption is successful, if limited. In the end, Scott accepts Obama’s judgment. ‘She was a very strong person in her own way,’ he tells her. ‘Resilient, able to bounce back from setbacks, persistent.’ After all, she finished her dissertation in anthropology despite a decade and a half of interruptions and delays. ‘But despite all those strengths,’ he concludes, ‘she was not a well-organised person.’ The judgment is withering but anticlimactic. Disorganisation, Scott observes, could mean anything ‘from a messy house to a messy life’. Obama replies: ‘All of the above.’ Dunham’s life was neither as destructive nor as self-destructive as her first husband’s. But it was, in many ways, as messy. And the chaos of both helped to shape the carefully calculating style of their son.

Dunham was a resourceful single mother attracted to exotic brown men. Having returned to the East-West Center in 1963 to study anthropology, she met Lolo Soetoro, a Javanese graduate student who was easygoing and patient – the opposite of Barack Sr. They were married in March 1964. Little more than a year later, after a right-wing coup brought the Suharto regime to power, Indonesians studying abroad were required to return for military service. Lolo complied, then sent for his wife and stepson. They joined him in 1967, after Ann had taken her degree. Barry was six years old.

The couple gradually veered into different social worlds. Barry’s stepfather took a middle management position at Union Oil; his mother, refusing the ornamental role of corporate wife, took a job teaching English and ‘business communications’ in a management school for non-governmental organisations, funded by the United States Information Service – a characteristic enterprise of the Kennedy era, intended to create a pro-Western Indonesian elite. Worlds apart, in Cambridge and Jakarta, both parents were participating in a common atmosphere of confident Cold War liberalism.

For Barry those years were anything but confident. Coming of age in Indonesia was hard for a skinny black kid. Racist hostility complemented conformist ideals of physical beauty. Once when he was nine, he was out for a walk with his mother and a friend. He ran ahead. Scott writes:

A flock of Indonesian children began lobbing rocks in his direction, ducking behind a wall and shouting racial epithets. He seemed unfazed, dancing around as though playing dodgeball ‘with unseen players’, [Ann’s friend Elizabeth] Bryant remembered. Ann did not seem visibly to react. Assuming she must not have understood the words, Bryant offered to intervene. ‘No, that’s okay,’ she remembered Ann saying. ‘He’s used to it.’

‘I think this is one reason he’s so halus [patient, calm, polite],’ Bryant told Scott. ‘If you’re not a good listener in Indonesia, you better leave.’ Indonesian schools fostered a culture of self-control; to survive one had to cultivate the ability to withstand taunting and teasing. As another of Scott’s informants says, ‘if you get mad and react, you lose. If you learn to laugh and take it without any reaction, you win.’ In Indonesia, Obama learned to be cool.

He also learned to be ambitious. His mother worked overtime at school and, Scott writes, ‘seemed barely to sleep. She would stay up, typing and correcting Barry’s homework, then be up before dawn.’ Co-workers and her houseboy Saman say Ann spanked Barry and even hit him with her father’s military belt; the president denies it. Whatever Ann’s methods, she encouraged the boy to cultivate grandiose goals. ‘What do you want to be when you grow up?’ Lolo asked one evening. ‘Oh, prime minister,’ Barry said. Bursting with pride over her bright boy, Ann used her parents’ connections to get Barry a scholarship to Punahou Academy in Hawaii.

The separation was hard for a ten-year-old boy, though he was going to a familiar place to live with his grandparents. But his mother and his baby sister Maya soon joined him. Lolo stayed behind while Ann returned to graduate school in anthropology at the East-West Center. Pursuing a PhD while she raised biracial children with different fathers was ‘unusual and dangerous and difficult’, Scott writes, even if she could depend on her parents for financial support. Her mentor was Alice Dewey (John’s granddaughter), who had been part of the Harvard team that included Clifford Geertz in Indonesia during the 1950s. For her thesis project, Dunham investigated cottage industries as a subsistence alternative for peasant families in Java. In 1975, Ann left for fieldwork in Indonesia, taking Maya with her. Barry chose to stay in Hawaii with his grandparents to finish high school. This was not abandonment, Scott insists; it was what the boy wanted, and the ‘single hardest thing’ his mother ever did.

Like other anthropologists, Ann gravitated toward the Population Studies Centre at Gadjah Mada University in Yogyakarta, where E.F. Schumacher came to lecture and small was still beautiful. Lolo, sick with liver disease, lived apart from Ann: their relationship ‘had ceased to be a functioning marriage’, Scott writes, though their divorce did not become final until 1980. Dunham, living on fried noodles and durian, the vile-smelling local fruit, plunged into research among the Javanese peasantry. She discovered that the industrialisation of agriculture had generated widespread rural unemployment, especially among women, and that Indonesian ‘backwardness’ was less a consequence of cultural values than of access to capital, or lack of it. Like her ex-husband in Kenya, she was learning to question the dogmas of development.

Returning to Hawaii for Barry’s senior year of high school, she was troubled by his apparent aimlessness. When she derided him as a good-time Charlie, the boy shot back: ‘Well, why not? Maybe that’s what I want out of life. I mean, look at Gramps [her father] … is that what you’re worried about? That I’ll end up like Gramps?’ The boy had become impatient, irritated by the sense that he was her ‘experiment’ – her effort, as Obama later told Scott, to create a son who was ‘sort of a cross between Einstein, Gandhi and Belafonte’. Rebellion was a brief fling. By autumn 1979 Barry was a freshman at Occidental College in Los Angeles.

Ann returned to take a job with the US Agency for International Development in central Java. She was less an anthropologist than a community organiser, supporting cottage industries, arranging the kind of small-scale lending that would later become known as ‘micro-credit’. This was the beginning of a series of jobs, later with the Ford Foundation and the People’s Bank of Indonesia, that placed her in the midst of the transition from government-sponsored development liberalism to market-driven neoliberalism. Throughout this chronicle, Scott underscores Dunham’s basic competence: ‘She set goals, met deadlines, was a team player, did not bend rules.’ As a co-worker says, ‘This notion that she was this hippie wanderer floating through foreign things and having an adventure is not the Ann I know … In a sense, she was as Type A as anyone on the team.’ She was a tough customer when insulted. ‘Don’t call me honey,’ she told a male contractor who had grown a little chummy. ‘Okay, sport,’ he said. However she acted around her children, Scott writes, there was nothing ‘soft and a bit naive’ about Ann Dunham.

Like her, Barry nurtured a sceptical liberal attitude toward imperialism in the guise of development. Having transferred to Columbia University and graduated in 1983, he wrote to her from New York that he was ‘working for the enemy’: Business International Corporation, a consulting firm supplying information about foreign countries to potential investors. Within a few months he would head for Chicago and more congenial work as a community organiser.

Ann continued to piece a life together in Indonesia, pioneering microfinance programmes and eventually completing a dissertation called ‘Peasant Blacksmithing in Indonesia: Surviving and Thriving against All Odds.’ Weighing in at more than a thousand pages, the study confirmed what she had been learning for years in the back-country of Java: village prosperity was constrained not by a lack of entrepreneurial spirit but by a lack of capital, not by culture but by political economy. At about the same time, the early 1990s, Barry graduated from Harvard Law School. So he’s going to be a billionaire, one of Ann’s friends said. No, Ann said, he’s going to return to Chicago and do pro bono work. ‘Okay, so he wants to be president,’ the friend said. ‘To his surprise,’ Scott writes, ‘Ann began to weep.’ When her son moved to Chicago and made what she saw as a ‘professional choice’ to identify as black, Dunham felt he was distancing himself from her.

A PhD at 50, Dunham pursued a career in microfinance that led her to a poorly paid job in New York at Women’s World Banking. She became the ‘village elder’ for younger colleagues: ‘She could be kind of a fat lady with big hair and a bohemian, but she was trying to do the right thing for the right reasons,’ her boss admitted. Life in New York was cold, hard and expensive. Soon she fled back to Jakarta for a better paying job with the Indonesian State Ministry for the Role of Women, but it didn’t last long. The stomach pains that had been troubling her for months continued even after an appendectomy. Eventually she returned to the US for a diagnosis of cancer and protracted, ineffectual treatments. In November 1995 she died.

By then Barry was Barack Obama, and was launched on a career of careful striving. His chief collaborator and confidante was Michelle Robinson Obama, an African American with roots in working-class Chicago, a fellow graduate of Harvard Law School and an attorney at the Chicago firm Sidley Austin. He had a summer job there in 1989; she was assigned to be his mentor. They married in 1992. When he ran for state senate in 1996, he softened her scepticism towards electoral politics by asking: ‘What if a politician were to see his job as that of an organiser … as part teacher and part advocate, one who does not sell voters short but who educates them about the real choices before them?’ Michelle was less confident than Barack that ‘systemic change’ could be brought about through politics, but she yielded to his organiser’s vision.

Jodi Kantor’s account of the Obamas shows how ill-suited they were to the suffocating social rituals of official Washington on their arrival in the White House, how jealously, and vainly, they tried to guard their privacy and freedom, and how disappointing the presidency has become – not only to the First Lady but to the president himself. They were outsiders, in their racial background, their eagerness to stay in touch with black friends from Chicago and their efforts to maintain a life with their two daughters. A desire for family dinners and evenings at home led to many turned-down invitations, which reinforced their isolation.

The president’s personal style had some of the same effect. ‘I’m not Bill Clinton,’ he said on the brink of declaring his candidacy. ‘I don’t need this [the adulation of crowds] … I don’t need anything.’ The conciliator, the community organiser, was also an isolato, withdrawing to the Treaty Room late at night to make his hardest decisions. At heart he was a technocrat rather than a politician. As a staff member said, Obama ‘believes that policy is a science (largely economics), that maximising the greatest benefits for the greatest number is the objective, and that there are “correct” and “incorrect” choices.’ He privately derided the pompous and powerful, and felt a natural attraction to underdogs, but was always convinced that he was the most serious man in the room. Still, when it came to his advisers’ flaws, he was forgiving. ‘The habit ran deep: he was the son of an arrogant, self-absorbed father who abandoned him and a loving mother who nonetheless sent her child to live across the ocean from her,’ Kantor writes. ‘To hold those around him to strict standards would have left him with no one, and he learned to rely heavily on himself to compensate for his parents’ failures and absences.’ These observations resonate with the image of the young Barry Obama, self-contained and alone in Indonesia, playing dodgeball with invisible opponents.

The combination of isolation and conciliation created the appearance – which became the actuality – of weakness. Michelle fumed. Though he wasn’t pickling his brain in alcohol, the son, too, seemed headed for a tragic fate: an exceptionally intelligent and decent man undone at least in part by his own contradictory temperament.

There was only one escape from the frustrations of Capitol Hill politics: foreign policy, especially with a military edge. Obama lunged eagerly toward the aggrandisement of executive power that he had so cogently criticised in Bush. Never was this more apparent than when he announced the killing of Osama Bin Laden: ‘His expression was cool, hard and satisfied: he had eliminated the country’s top enemy, accomplished what George W. Bush could not, and finally shown the country the utterly serious matters he had been dealing with,’ writes Kantor. No doubt about it: ‘There was something disciplined and satisfying about secretive national security work. There was no messy Congress to deal with, no stroking and horse-trading with legislators, and certainly no Tea Party resistance.’ Obama’s preference for Navy Seals over Republican politicians may be understandable, if we are talking about who sits on the bar stool next to yours, but for a president the implications are alarming.

We are back in the world of the steely-eyed commander in chief. Recent policy reflects the revitalisation of the Washington interventionist consensus. The demonisation of Iran accelerates, while the president dispatches US troops to Australia and the secretary of state to Burma. The open door for US involvement in Asia, flung wide in Japan’s face a century ago, is now reopened in China’s. One can only imagine the American reaction, were China to make a similar move in Venezuela or Colombia. Obama’s recoil from disappointment may turn out to endanger us all.