The 2016 presidential election took a heavy toll on the vast army of journalists assigned to cover it, grinding down shoe leather, fingertips and nerve-endings in equal measure. But for one reporter, Daniel Dale of the Toronto Star, the race for the White House was singularly burdensome, turning him into a night owl.

At the end of each long day on the campaign trail, he would take a deep breath and launch into his second job: fact-checking the lies of Donald Trump. The work would begin late, often at 2am, when all was quiet and he could sink himself undisturbed into a hot bath of outrageous falsehood.

He’d start by watching recordings of the rallies the Republican candidate had addressed that day, scouring his words for deceits that ranged from the whopping (“I opposed the Iraq invasion”; “the election is rigged”) to the trivial (“the only way to pronounce Nevada is Nev-AHH-da”, when in fact most people say “Nev-ADD-a”.)

Then the reporter would scuttle through Trump’s Twitter feed, followed by news outlets and search engines in the hope of catching every statement he made and testing them all for accuracy. As the election approached, Trump’s untruths began to multiply, and with them Dale’s nocturnal hours, until he was at it virtually all night. After each of the three televised debates between Trump and Hillary Clinton, and in the final frantic days of the campaign, he pulled all-nighters before going straight back to his day job.

“Towards the end it was crazy,” Dale says. “The only thing that made it easier was that Trump repeated himself: we called him out for lying but he was so unresponsive he just kept saying the same things.”

From 15 September until 8 November, Dale recorded a total of 560 false Trump statements, an average of about 20 a day. As the sleep-deprived reporter staggered into election day, he reflected that if his nighttime activities had reached even a small percentage of the American electorate, and helped them understand the fundamental mendacity of a candidate seeking the most powerful job on the planet, then “it would have been worth it”.

Then came election night. “I was as shocked as many other journalists,” Dale says, recalling Trump’s steady march towards the 270 electoral votes needed to win. Despite his own extraordinary late-night efforts; despite the similarly herculean efforts of the New York Times, which dedicated 18 journalists to fact-checking the TV debates in real time, or of NPR, which turned over 30 staffers to a similar endeavor; despite the Guardian’s Lyin’ Trump column and so much more, some 61 million Americans were unfazed enough by the idea of a serial liar in the Oval Office to vote for him.

Fact-checking wasn’t the only journalistic tool that was deployed in 2016 with what now appears to be limited effectiveness. Many observers assumed the Washington Post’s thundering scoop of Trump’s hot-mic remarks in which he boasted about grabbing women “by the pussy”, together with Megyn Kelly’s astonishing opening question at the first TV debate in which she reminded the candidate that he had called women “fat pigs” and “dogs”, would hand female voters, and thus the presidency, to Clinton. They didn’t – 53% of white women voted for Trump, according to exit polls.

Jorge Ramos, the lead news anchor at Univision, predicted on election day that Trump would lose because he had turned his back on the Spanish-language media. Didn’t happen – almost one in three Hispanics backed him, to Ramos’s bafflement.

Accomplices or antagonists: how the media handled the Trump phenomenon Read more

Newspaper endorsements, those most portentous of journalistic institutions, also seemed to have little purchase on the outcome of this convention-shattering election. The Nieman Lab counted 360 titles that backed Clinton, including the Dallas Morning News which sided with a Democratic candidate for the first time since 1940, and USA Today which endorsed the first presidential candidate, period, in its 36-year history. By contrast, Trump drew a paltry 11 endorsements. Such overwhelming consensus from the nation’s press – but was anybody listening?

Put all these indicators together, and you start to wonder whether Donald Trump’s unlikely victory has sounded the death knell for the influence and authority of what he and his supporters scathingly call the “mainstream media”. Did the “MSM”, in particular cable TV which broadcast his every cough and spittle with almost obsessive dedication, help put him in the White House? Conversely, did the press go too far in abandoning its traditional even-handedness and unrestrainedly attacking Trump, as rightwing pundits are now suggesting? Looking forward, what will be the role of the established media as we head into the chilly waters of a choppy new era?

Those are just some of the glaring questions left hanging at the end of an epically weird election season. To begin poking some of those issues, the Guardian has teamed up with the Columbia Journalism Review (CJR), a leading source of media criticism.

It has compiled a unique oral history of the 2016 election through the eyes of 40 of the top TV and press editors, reporters and columnists who helped shape and define the public perception of this year’s race. The narrative runs to 11,000 words, with the Guardian publishing an edited version of it.

One of the first big questions that CJR’s oral history, in tandem with the Guardian’s own exploration of the subject, throws up is the role of blanket TV coverage in the rise, fall and rise again of President-elect Trump. Ben Smith, editor-in-chief of BuzzFeed, was quick to spot earlier this year that Trump’s Twitter feed acted not only as a powerful megaphone channeling his voice directly to his followers (he had 13m of them on Twitter on election day, as well as 12m on Facebook), it also served as a news script for the cable channels. Trump would tweet out his latest offensive or aggressive tweet, often in the early hours of the morning, and TV news editors arriving at their desks would pick it up and turn it into ratings-friendly headlines that would then go on to dominate the rest of the 24-hour news cycle.

“What typically drives TV is great reporting by newspapers. Instead, Trump used his tweets as an alternative route to great journalism, and that turned out to have been a very big deal,” Smith told the Guardian.

The TV channels feasted on tweeted Trumpisms with an alacrity that many observers found disturbing. According to a study by mediaQuant, Trump benefited from the equivalent of $5.2bn worth of free airtime from earned media, a quantity that the firm’s analysts said was “almost surreal in sheer magnitude”. In return, cable channels enjoyed a massive ratings spike.

TV was convulsed by other controversies, notably CNN’s decision to employ Trump’s former campaign manager Corey Lewandowski while he was still receiving “severance payments” from his old boss, and the embarrassment (also falling to CNN) of Democratic strategist Donna Brazile leaking questions from a town hall event in advance to the Clinton campaign. More red faces erupted at NBC when its Access Hollywood tape of Trump’s lewd comments about women was leaked to the Washington Post after the network appeared to dither over broadcasting it.

But these difficulties, humiliating at times though they were, pale in comparison with the overarching charge that TV – and to a lesser degree the press – was indiscriminate for far too long in its handling of Trump, engaging with him more like the Apprentice reality TV star that he was than the US president that he wanted to become. “During the primaries, cable news – in particular CNN and Fox, and to some extent MSNBC – gave Trump unhealthily generous, inflated coverage. That helped make him,” says Margaret Sullivan, media columnist of the Washington Post.

It takes someone with the on-the-ground experience of John Weaver, chief strategist to Trump’s rival John Kasich during the Republican primaries, to spell out the full impact of this media binge on the election. He believes it actively distorted the process, and even now, six months after Kasich dropped out of the race, you can still hear the fury in Weaver’s voice.

“I hope there’s some deep introspection at the networks and cable channels over the billions of dollars of free coverage they gave to Donald Trump, often without holding him accountable for his bizarre claims. He is judged by such a low standard by many of the news media, it’s troubling to me.”

The sense that the media allowed itself to be taken for a ride by the Trump campaign in exchange for stellar ratings came to a crunch on 16 September in Washington DC. By now enshrined as Republican nominee, Trump lured the pack of political reporters to a press conference by promising to make a statement – an apology, it was assumed – about his long-held and oft-repeated “birther” lie about President Obama having been born outside the US.

The location for the event happened to be the Old Post Office on Pennsylvania Avenue that Trump had just renovated into a luxury hotel. For 26 minutes he subjected the media, and through them the American people, to a succession of unctuous speeches by retired military leaders singing the candidate’s praises. Then, when Trump finally got round to issuing a 21-word rebuttal of his own false birther theory, he prefaced it with yet another lie that Hillary Clinton was responsible for having started the toxic rumor in the first place.

It was vintage Trump. Almost a half-hour of free advertising – not only for his presidential campaign but also for his new hotel – handed to him on a plate by the cable networks in exchange for a 30-second non-apology that made false accusations against his opponent. Nothing new there, then.

What was new was the response of the assembled media crowd who had been forced to sit through the ordeal, cameras rolling, keyboards rattling, as though they were so many of Trump’s dutiful playthings. It was a “political Rick-roll” fumed CNN’s Jake Tapper, referring to the Rick Astley internet meme, while John King on the same channel lamented: “We got played again.”

Jay Rosen, who teaches journalism at New York University and whose blog PressThink.org has been a widely cited resource on the fraught relationship between Trump and the press in 2016, sees the DC hotel debacle as a seminal moment. “That’s when people in the campaign press corp got disgusted not just with Trump’s mendacity and manipulation but at themselves for playing along with it. That turned the worm.”

“It was an important turning point,” agrees Kyle Pope, CJR’s editor and publisher. “It’s when reporters said, ‘No more, that’s it, enough’.”

The response from the New York Times was decisive and swift. As its political editor Carolyn Ryan related to a recent CJR roundtable of media executives, she immediately approached the paper’s executive editor.

“I went to Dean Baquet and I said, ‘Can we call this the unwinding of a lie?’ He understood the potency of coming out very directly and saying that, and that what Trump had done – a willful deceit – was far more egregious than some of the fabrications and stretching of the truth that we usually see in politics. So he was firmly behind us.”

The resulting piece by Michael Barbaro marked in itself a seminal moment for the Times newsroom, which as Ryan pointed out is a “fairly traditional institution” that until recently struggled with the idea of calling a lie a lie. Yet in this case the editorial decision to do just that – abandoning any attachment to the “on the one hand this, but on the other hand that” school of journalism – came quickly and easily. “We decided in about 40 minutes,” Ryan said.

There has been a lot of discussion this election cycle about how Trump’s insurgent campaign, its asymmetry as Rosen has put it, has shaken up many of the time-honored rituals and false equivalencies of political journalism. Adam Moss, New York magazine’s editor-in-chief, makes clear that his magazine consciously decided to ditch the journalistic habits of a lifetime: “We don’t pretend to objectivity, but we do, as a rule over the years, try to maintain some kind of even-handedness. We abandoned that because we felt the threat of his candidacy and presidency was too great.”

The counterpoint to that is that several of the contributors to the CJR oral history express regret that, at least at the outset, the coverage of real estate businessman was “seriously flawed”, as Jorge Ramos puts it. The billionaire was underestimated as a candidate, and as a result treated initially as a bit of a joke. Laura McGann, a politics editor for Vox, bravely admits feeling guilty that she assigned a deep dive into Trump soon after he launched his campaign in June 2015 to a (very good) intern, adding: “I regret not taking him more seriously a year ago than I did.”

But BuzzFeed’s Ben Smith makes the point that, important though such post-election soul-searching is, it should not obscure the fact that there was plenty of hard-hitting reporting done even early on. “The Washingon Post, New York Times, we at BuzzFeed, Politico, the Guardian and many more, gave a thoroughly detailed account of him. Anyone who was surprised by the person who emerged during the campaign really wasn’t paying attention.”

You only have to credit a tiny portion of the highlights of the coverage this cycle to see how impressive much of it was: Mckay Coppins’ seminal romp with Trump all the way back in 2014; Susanne Craig’s heaven-sent package in her Times mailbox containing the billionaire’s leaked tax returns; David Fahrenthold of the Post’s work on the Trump Foundation and his groping scoop; Ari Berman of the Nation’s probe into the real rigging of the election; Guardian columnist Lucia Graves’ interviews with women alleging the candidate sexually assaulted them.

So what happened to all this solid work? Why did it appear to go up in a puff of smoke on election night? There is a separate and febrile debate over whether or not opinion polls were in part to blame for giving the impression that the White House was in the bag for Clinton, but many other theories are circulating.

One prevalent idea, as Hadas Gold of Politico tells CJR, is that the media did its job but the public “just did not care”. The New York Times political reporter Jonathan Martin offers an opposite conclusion: that the coverage did hit home with voters, as reflected in Trump’s historically bad popularity ratings. To which it might be added that Hillary Clinton is still winning in the popular vote.

A third line of inquiry that is likely to run and run is whether the media, in its coastal bubble, failed to get to grips with the anger that was brewing right under its nose in the American heartlands. A flick of that argument was given in the missive sent by Arthur Sulzberger and Baquet, publisher and executive editor of the New York Times, to their readers soon after the election. “Did Donald Trump’s sheer unconventionality lead us and other news outlets to underestimate his support among American voters?”

Answering his own question, Baquet told the Guardian US that the Times did in fact write extensively about anger in America in the run-up to the election, but adds that it’s a good idea as an editor to be humble enough to ask oneself what questions should have been asked during the campaign.

“Nobody, including the Trump campaign itself, quite understood the forces that would lead to Donald Trump being elected in what everybody agrees was a surprise. A big part of my job now is to explain that in the coming months, to go out in the country, talk to people who feel anxious about the future of the country, don’t caricature them, let them speak. Just let them talk.”

A more disturbing thought is that while many news outlets took great efforts to chronicle the plight of the white working classes in middle America, their dispatches didn’t register with millions of voters for the simple reason that those voters weren’t reading any of us. Amid the ongoing muscle of Drudge, the new transcendence of Breitbart, not to mention the proliferation of fake news sites, conservative Americans – and for that matter liberals – are increasingly able to receive information from within their own alternative bubble where their opinions are reinforced without being challenged by the loathed MSM.

One of the more despairing expressions of this theory was given by New York magazine’s Adam Moss to CJR: “The media as it used to be thought of is just not that important any more. It doesn’t matter that some people were doing good work because … we’re just talking to ourselves and people who already agree with us.”

Since Trump’s victory, another theory has been aired increasingly by right-of-center pundits such as Howard Kurtz of Fox News that flips the argument that the media did too little too late to expose Trump on its head. In this iteration, major outlets did too much, were too eager to go after him. The editor of the Wall Street Journal, Gerard Baker, tells CJR that “some reporters saw it as their role to stop this man from becoming president, they put themselves in the role of partisans”.

It fell to Piers Morgan of Mail Online, as it often does, to put the case most stridently: “The NYT’s ludicrously biased anti-Trump barrage of bile helped him get elected.”

“That’s ridiculous,” ripostes the Times’ Dean Baquet. “We didn’t cover Donald Trump any more aggressively than we covered Hillary Clinton. People forget that we broke the story of Clinton’s emails. If both sides think you were tough on them, maybe that means you were fair.”

Of all the mainstream media bashing that has been unleashed since 8 November, one source of it stands head and shoulders above the rest: Donald Trump himself. Which is not surprising, given his track record during the campaign.

Reporters deemed to be producing unfavourable copy were punished by Trump: Jorge Ramos was ejected from a press conference after the Univision anchor dared to ask an awkward question; Coppins was turned away from Trump rallies as comeuppance for his caustic 2014 profile; a lengthening list of other media outlets were banned by the campaign; individual journalists were targeted for bullying, such as NBC News’s Katy Tur; supporters at campaign stops were enabled to turn in anger against camera operators just doing their jobs, screaming: “CNN sucks! CNN sucks! CNN sucks!”

Since Trump’s metamorphosis into president-elect, the tendency to target the media has only got worse. Now he routinely gives the press pool the slip, showing contempt for both transparency and tradition. Back on his Android phone, Trump is on the “failing” New York Times warpath, displaying a sort of frenzied disdain that hardly bodes well for the relationship between soon-to-be president and the country’s paper of record.

“I’ve never seen anything quite like this,” says Baquet, who as former editor of the Los Angeles Times and Washington bureau chief of the Times has seen more than most. “I don’t think we’ve ever had a president-elect tweeting about us, that’s for sure.”

As 20 January 2017 and the start of his presidency bears down upon us, much of the media will enter the Trump era carrying with them a profound sense of trepidation. “This is a moment of high danger for the press; we’re heading into a dark period for American democracy and American journalism,” says Jay Rosen of NYU. Jorge Ramos tells CJR: “I think we’ll remember this election as unique and very concerning for the future of the press because if this becomes the norm, we are in serious trouble.”

It’s not just the lack of access, mocking and bullying that can be expected from President Trump. There’s also the internal danger that the media will normalize his time in office under the cloak of traditional reverence for the presidency.

Contributors to the CJR’s account of 2016 point out that a pattern of normalization was established even on the campaign trail, when media outlets appeared to be willing Trump on to elevate himself from his startlingly unconventional approach to a proper “presidential” mode. In the memorable description of Tommy Craggs, Slate’s politics editor, the candidate “could have gone up there and danced the hoochie coochie and they would have said, ‘This is a welcome change in tone from Donald Trump’”.

Since the election that trend has thickened, some believe. John Weaver, the former chief strategist for Trump’s Republican rival John Kasich, says he is outraged by media references to Steve Bannon, the Breitbart chairman who Trump has appointed his senior counselor in the White House, as a “controversial figure” or an “aggressive conservative”. “No, no, that’s sugar-coating. He was the publisher of the news site that he said was home to the alt-right, and that is code for racism and sexism and fascism,” Weaver said.

So what will be the role of the “mainstream media” as it embarks, somewhat battered but still standing, into Trump’s America? Will it be to document the new administration with a detached and “objective” eye, as traditional newsroom canons dictate, or will it pursue that other burning function of the fourth estate, holding power to account?

“We will cover him fairly and aggressively, and we will not let his criticism of us sway us or keep us from doing what we have to do,” says Dean Baquet of the New York Times. “In my mind our mission is clearer than it’s ever been: we have to cover the giant story of this extraordinary White House, which is going to change American politics and policy for years to come, as aggressively, truthfully and honestly as we can.”