The former prosecutor and deputy attorney general has grappled with a series of contentious issues, including surveillance and the ‘Ferguson effect’

Three days ago, James Comey was one of Donald Trump’s symbols of Washington corruption and a law enforcer of “the highest integrity” to Hillary Clinton’s campaign. On Sunday, the FBI director was the most controversial figure in the capital, heroic to Trump, irresponsible to Clinton and puzzling to nearly everyone.

Comey has a long history of trying to walk political tightropes, to varying success, dating to the earliest days of his career.

Clinton and Trump camps duel over FBI director's late email revelation Read more

Comey, a career prosecutor who grew up in New Jersey and studied religion and chemistry, had his first brush with a high-profile investigation came in 1996, after a stint with the US attorney for New York. That year, he joined a Senate investigation into Bill and Hillary Clinton’s real estate investments, named Whitewater after a failed venture they joined.

He remained in the background of an investigation that ultimately petered out, but Comey soon joined two prestigious prosecutor’s offices, in eastern Virginia and Manhattan, where he pursued high-profile cases against identity thieves, a top Credit Suisse banker and Martha Stewart.

In 2003, he was named deputy attorney general in the administration of George W Bush, becoming a dissenting voice against surveillance programs of the National Security Agency revealed in 2005 by the New York Times.

Comey later testified to Congress that one night in March 2004, with attorney general John Ashcroft hospitalized for an emergency procedure, he raced to the sickbed to prevent the renewal of warrantless wiretapping programs which the justice department had determined were illegal.

Comey arrived just before Bush’s top lawyer and chief of staff, and Ashcroft did not sign the renewal. “I was angry,” Comey told the Senate in 2007. “I thought I just witnessed an effort to take advantage of a very sick man, who did not have the powers of the attorney general because they had been transferred to me. That night was probably the most difficult night of my professional life.”

The next day, Bush modified the wiretapping program.

In those hearings, about possible misconduct by the White House chief counsel and, ironically, missing emails from the Republican National Committee, Comey told Congress he felt it imperative that the justice department stand outside partisan politics.

“The Department of Justice, in my view, is run by political appointees of the president,” he testified. “But once they take those jobs and run this institution, it’s very important in my view for that institution to be an other in American life.

“My people had to stand up before juries of all stripes, talk to sheriffs of all stripes, judges of all stripes. They had to be seen as the good guys, and not as either this administration or that administration.”

Comey left the justice department in 2005 for Lockheed Martin, the largest military contractor in the US, and eventually an investment firm and Columbia Law School. In 2013, Barack Obama nominated the registered Republican to lead the FBI, joking that the 6ft 8in prosecutor was “a man who stands very tall for justice and the rule of law”.

He was confirmed as the agency’s seventh director in a 93-1 vote – only Kentucky senator Rand Paul voted against him, over domestic drone surveillance – and quickly took on a series of controversial cases.

Will Hillary Clinton lose the election because of the FBI's email investigation? Read more

Though he opposed the NSA’s warrantless wiretapping, Comey has emerged as a steady advocate of US security agencies since the Guardian revealed a host of programs in 2013, a system that gave the FBI access to a trove of data. Comey has repeatedly argued that security agencies should be given tools to bypass private encryption, and took the FBI to court against Apple over unlocking the phone of one of the San Bernardino terrorists.

“We’re asking Apple to take the vicious guard dog away and let us pick the lock,” he told Congress in March. “It’s not their job to watch out for public safety. That’s our job. The logic of encryption will bring us to a place in the not too distant future where all of our conversations and all our papers and effects are entirely private.”

The legal battle ended in anticlimax, as the FBI found another way into the iPhone. It has since said that the hack it used does not work on newer phone models.

Comey has walked a careful line on race and police killings. Last year he said it was “unacceptable” that the Guardian and Washington Post had better data on police shootings than the federal government, but also dipped into controversy by suggesting a “Ferguson effect” – named after sometimes violent protests in Ferguson, Missouri, over the police shooting of an unarmed black teen – may cause police officers to step back from their responsibilities.

Comey conceded he lacked evidence for that claim, which led him into an unusually public disagreement with the president, who warned last year: “What we can’t do is cherry-pick data or use anecdotal evidence to drive policy or to feed political agendas.”

Since then, tentative research by the justice department and a St Louis criminologist has suggested the “Ferguson effect” is plausible as an explanation for a single-year spike in violent crime.

The director rose to national prominence in July, when he announced the FBI’s findings in its investigation into whether Clinton and her staff had acted criminally in their use of a private email server. He concluded there was no evidence of intentional wrongdoing.

Comey used a press conference – itself a break from tradition and his “original sin”, in one former official’s words – to lay out the facts at the time, correct several false claims by Clinton, and castigate her and her staff for their “extremely careless” email practices. He was then called to Congress to defend his decision, which ended months of expensive and inconclusive Republican investigations.

“You know what would be a double standard? If she were prosecuted for gross negligence,” Comey told the House. “No reasonable prosecutor would bring the second case in 100 years based on gross negligence.”

“It’s not fair to prosecute someone on these facts,” he insisted, calling the procedure “celebrity hunting” and adding: “One of the things I have learned in this case is that the secretary may not have been as sophisticated as people would assume.”

Asked about his voter registration, Comey said he no longer belonged to any party.

Kellyanne Conway: Trump fan chanting ‘Jew-S-A’ behaved in ‘deplorable’ way Read more

The short letter Comey wrote to congressional leaders on Friday admits that he does not know whether new emails found on a computer belonging to the estranged husband of a Clinton staffer have any significance. That lack of information has turned attention to the director himself, who reportedly broke with not only precedent but advice from top justice department officials by going public.

An internal memo Comey sent to staff, leaked to several news organizations, suggests the director felt he had no choice but to act, given his testimony to Congress.

“I also think it would be misleading to the American people were we not to supplement the record,” he wrote. “At the same time, however, given that we don’t know the significance of this newly discovered collection of emails, I don’t want to create a misleading impression.

“In trying to strike that balance, in a brief letter and in the middle of an election season, there is significant risk of being misunderstood, but I wanted you to hear directly from me about it.”