As the first polls closed on 8 November, Hillary Clinton was preparing for the moment she would at long last take the stage as the first female president of the United States.

She instead found herself apologizing to Barack Obama hours later in a harrowing phone call, moments after conceding the presidential election to Donald Trump.

“Mr President, I’m sorry,” Clinton told Obama, who had contacted her to extend his condolences for her loss.

The phone call is one of several revelations in the new book Shattered, in which the journalists Jonathan Allen and Amie Parnes chronicle Clinton’s 18-month quest to do what eluded her in 2008 and break “that highest, hardest glass ceiling”.

The book sheds particular light on the painful turn of events on election night, as Clinton watched the returns deviate dramatically from the path her campaign had so confidently predicted.

Surrounded by family and aides at the Peninsula Hotel in Manhattan, she saw the first cracks emerge as the must-win state of Florida turned dicey. Steve Schale, among the most reliable Democratic strategists in the state, informed Clinton’s campaign they were short on votes.

As other battlegrounds began to crumble, a sober mood took hold in the suite. But soon, the authors wrote, aides began to blame each other as Bill Clinton grew increasingly upset and Hillary Clinton became withdrawn.

It was Obama, taking in the results with his own advisers in Washington, who broke up the slumber with a series of phone calls from the White House. Wisconsin, a Democratic stronghold, had just been called for Trump. Some states remained excruciatingly close, but the president was convinced of the outcome and did not want to question the legitimacy of the election as Trump had done just weeks earlier while facing off against Clinton in the debates.

“You need to concede,” Obama told Clinton after his political director had failed to persuade Clinton’s campaign manager, Robby Mook, on the point.

It took one more phone call from the president – to Clinton’s campaign chairman, John Podesta – before she asked her close aide Huma Abedin for the phone and uttered the words she never thought she would say: “Congratulations, Donald.

“I’ll be supportive of the country’s success, and that means your success as president,” Clinton said.

It was after her brief conversation with Trump that Clinton apologized to Obama.

The book is the second collaboration for Allen and Parnes, who in 2014 authored HRC, a detailed account of Clinton’s tenure at the helm of the state department, in anticipation of her second presidential run.

Hillary Clinton addresses her staff and supporters after losing the presidential election. Photograph: Carlos Barria/Reuters

As the first take on Clinton’s doomed campaign, they offer a behind-the-scenes view of the obstacles in her way – some familiar and others a consequence of the shifting American electorate.

There is the familiar tension between the old guard that has for decades operated “Clintonworld” and the new blood that labored in Brooklyn to reinvent the campaign of one of the most well-known public figures in the world.

The authors revisit the unexpected challenge from Bernie Sanders, whose insurgent campaign captured the wave of economic populism sweeping the country. The infighting amid the Democratic primaries, as Team Clinton struggled to decisively put away the senator, is a dominant theme of the book’s first half.

And then there are the emails.

The aftermath of Clinton’s decision to use a private email server as secretary of state, and subsequent discord among her inner circle over how to respond, is retold with the benefit of hindsight. We learn that Bill Clinton did not believe she should apologize, as neither he nor she felt there had been any wrongdoing.

Clinton eventually did apologize to the public in August 2015, six months after the server issue was first brought to light.

The role of Russian interference in the US election, now the subject of three investigations in Washington, is barely explored in the book, although the authors do say that Clinton’s former aides see it as one of the main reasons for her loss.

The effects of Trump’s nationalist campaign are also relegated to the backseat, as this is a story primarily told through how Clinton’s campaign adapted (or sometimes crucially didn’t) to the evolving political climate. Staffers were confident in their sophisticated voter modeling and analytics, and fatally failed to spot when they veered off target.

Clinton, for her part, carried a lingering doubt throughout the campaign.

“Why aren’t they with me?” she asked of working class white people during the Democratic primaries. “Why can’t we bring them onboard?”

It was a question her campaign never quite answered.