Last Christmas Eve, Ken Starr took his family to a cosy restaurant in New York’s West Village. They had finished their meal and were leaving for a carol service when he spotted in the foyer a dark-haired woman who looked familiar.

“She bears a remarkable resemblance to Monica,” he thought, recalling the White House intern who had been the focal point of the special investigation he spearheaded as independent prosecutor two decades ago, which culminated in the impeachment of a US president. Then the penny dropped: it was her.

Remarkably, until that moment, Starr had never met Monica Lewinsky, the woman he put on the rack in 1998 as he tried to extract evidence from her about her tryst with Bill Clinton. According to Lewinsky’s account of the awkward encounter, she managed to stammer to Starr that she wished they had both made different choices back then, as a gentle way of inviting him to say sorry to her. But he made no move to apologise that night. The two exchanged pleasantries and went their separate ways.

Now, nine months later, Starr, 72, has a second chance to set things right with Lewinsky. We are discussing Contempt, his new memoir of the Clinton investigation, when I put the question to him: “Do you owe Monica Lewinsky an apology?”

I choose my words carefully, repeating verbatim the question NBC News posed to Clinton in June. Outrage ensued when the former president blurted: “No, I do not.” Starr’s response to the same question is identical, surprisingly so given that the two men were arch-enemies. “No,” he tells me, adding the non-apology: “I regret the sorrow she went through, the travail she went through.”

It is 20 years this week since the Starr report was published, tearing a massive rip in the political fabric and leading to only the second impeachment of a US president in history. The report, and the 11 charges against Clinton that went with it, propelled Starr into international notoriety as the Republican scourge out to get a popular Democratic leader. He was dubbed, as he recalls in the book, “the most criticised man in America”. Halloween masks were cast of him.

So it’s with some trepidation that I welcome this fearsome figure into the Guardian offices in New York. When he arrives, wielding a large umbrella and generous smile, he comes across as startlingly harmless. He is wearing a pinstripe suit and baseball cap, a clash of styles that presumably passes as normal in his home state of Texas, but gives him the avuncular air of a small-town bank manager.

When we sit down, I remind Starr that what he did on 16 January 1998, in an operation dubbed “Prom Night”, was anything but harmless. He arranged for Lewinsky to be hustled by a group of male officers to a hotel room where she was threatened with 27 years in prison – three more years than her age at the time – unless she wore a wire and snitched on Clinton. All because she had lied about her affair with the president in a civil lawsuit.

Starr went on to reveal her most intimate details, semen-stained blue dress and all, to the entire planet. He turned her life, as she put it, into a “living hell”.

No apology?

“We gave her the opportunity at the Ritz Carlton hotel in very pleasant circumstances to cooperate with the investigation and she chose not to,” he says. “It was a very unwise decision.”

It was bullying, I say.

“I completely reject the suggestion there was bullying. We were clear and emphatic: you’ve engaged in very serious misconduct, do you want to cooperate or not?”

It’s uncanny how the Starr phenomenon is back in the headlines 20 years later. It’s not just the publication of his book, a thinly disguised justification for his hounding of the Clintons over the Lewinsky affair and in the earlier investigation into an obscure Arkansas land deal called Whitewater.

Hillary Clinton watches her husband as he thanks members of the US House of Representatives who voted against his impeachment in 1998. Photograph: Susan Walsh/ASSOCIATED PRESS

It’s also that #MeToo has brought the affair of the president and the intern in the Oval Office back into view as the mother of all gender-related workplace abuses. Then there’s the fact that the White House incumbent is once again in the sights of a special prosecutor – for Ken Starr 1998, read Robert Mueller 2018.

Several lead characters have made a comeback, from Hillary Clinton’s failed 2016 presidential bid down. Brett Kavanaugh, who was a key member of Starr’s team in the Clinton investigation, is on the verge of being confirmed to a supreme court seat that would lock in rightwing control of the nation’s highest court for generations. Rod Rosenstein, another core Starr investigator, is the justice department official overseeing Mueller’s dive into possible collusion between Donald Trump’s campaign and Russia.

“There are eerie echoes of the past,” Starr says. “The years go by yet many of the characters are back on stage, and some of the issues – presidential accountability, rule of law – have re-emerged.”

So what tips would Starr have for Mueller? “My advice would be what I told my team every day: ‘Let’s just get this job done as quickly and professionally as we can. We are being very closely watched.’”

Mueller appears so far to have avoided traps that befell Starr. Mueller is discreet, focused, ruthlessly effective – adjectives rarely attached to his predecessor in the Clinton days. Could it be that Mueller studied and learned from his mistakes?

Starr thinks so. He shudders as he recalls giving a couple of high-profile media interviews to disastrous effect. ABC’s Diane Sawyer snarkily wondered on live TV whether she had the right to ask him if he had cheated on his wife (he said he hadn’t).

By contrast Mueller has kept schtum. “He’s got into a submarine and gone deep and silent into the depths of the investigation,” Starr says. “I have no doubt he is mindful of the past and trying to avoid landmines.”

As for Trump, Starr says he is struck by the “overwhelming distinctions” between the president and Clinton in their responses to their respective investigations. He tells me that, from what he has seen, Trump “has not engaged in obstruction of justice and has been cooperating, whereas the Clintons did everything they could to be obstructive”.

That’s a bold statement given Trump’s firing of the FBI director James Comey and the increasingly bizarre attacks on Mueller by Trump’s lawyer, Rudy Giuliani.

Monica Lewinsky at the 2017 Oscars. Photograph: Danny Moloshok/Reuters

You could equally turn the observation back on Starr: there appears to be an overwhelming distinction in the standards he applied to Bill Clinton and those he now applies to Trump. In the 1990s he showed Clinton no mercy – pursuing him relentlessly over five years: from Whitewater, to Vince Foster’s suicide, “travelgate”, “filegate”, “troopergate” and finally Lewinsky.

In the end, it came to nothing. His team discussed charging Hillary Clinton but concluded they lacked proof. After Bill Clinton was impeached, in February 1999 he was acquitted on all counts by the US senate.

Starr also gives Trump the benefit of the doubt on perjury, saying: “To my knowledge, he has not lied under oath.” That’s a telling comment, given what he did to Clinton.

At the point when Lewinsky was grabbed by Starr’s men on Prom Night, Clinton had not yet lied under oath. It was only the following day, after Clinton, ignorant of what had just happened, was deposed in the Paula Jones lawsuit that he perjured himself by lying about the relationship. That sounds to me like entrapment, I say. Why didn’t Starr simply pick up the phone and call the White House to ask what was going on with Lewinsky rather than allow the president to hang himself?

“Because we had seen this pattern of conduct before,” Starr says. “We felt he had been completely untruthful with us, deliberately so.”

There are two ways of looking at Starr’s contrasting approaches to Trump and Clinton – Trump innocent until proven guilty, Clinton guilty as hell so let’s find the evidence to prove it. You could say he’s just being a lawyer, or you could say the disparity leaves an acrid, partisan taste in the mouth.

It’s precisely that taste that is making Democrats so anxious about Kavanaugh as he gets close to being confirmed to the supreme court. Kavanaugh was the lead writer of the 11 counts of impeachment against Clinton, a task he undertook with alacrity. Yet since 1998 Kavanaugh has changed his spots, increasingly favouring strong executive power in the face of prosecutorial challenges. That has prompted speculation that he might go easy on Trump and allow him to avoid subpoena in the Russia investigation.

Should we be worried? “I think everyone should rest easy,” Starr says. “Brett Kavanaugh is a lawyer’s lawyer, a judge’s judge.”

During Kavanaugh’s confirmation hearings, a 1998 memo he had written for Starr surfaced in which he sketched out questions to ask Clinton about Lewinsky before the grand jury. In it Kavanaugh castigated the president for his “callous and disgusting” behaviour, then posed questions that can only be described as, well, callous and disgusting.

Which brings us back to Lewinsky and the publication of the Starr report 20 years ago. When it was released by Congress on 11 September 1998, the US’s collective jaw dropped from all the lurid detail it contained. Rereading it today, you can see why. There’s the famous cigar reference, and precise accounting of how many times Lewinsky reached orgasm and when the president of the United States did or did not ejaculate. The document reeks of what Sawyer in that same ABC interview called “demented pornography for puritans”.

I was surprised that a lifelong evangelical Christian such as Starr would resort to such language, and I ask if he regrets it.

“I very much regret that Congress decided to release the report, we had not anticipated that. But we had to prove the case, the president had lied under oath.”

Weren’t you being prurient?

“No, that’s an unfair, unjust charge. We were prosecutors, we had to prove our case. Of course it was ugly. The president was warned that this would be embarrassing, then he was found in contempt because of his willing mendacity.”

The Starr report marked a crucial juncture in the postwar journey of the US, from the God-fearing family-oriented 1950s to the present day, when a man can boast about grabbing women by the pussy and still get elected president. Does he think that by dwelling so graphically on private sexual matters he contributed to this coarsening of public discourse?

For the first, and only, time in our hour together, Starr concedes. “I don’t think it was helpful in terms of public discourse, of course,” he says.

“I don’t think it was helpful.” It’s not the apology Lewinsky wants. But coming from the man whose name was once a byword for partisan intransigence, perhaps it’s a start.

Contempt: A Memoir of the Clinton Investigation by Ken Starr (£23.99, Penguin Random House) is out now.