The arrest of Robert Durst, the infamous real estate heir linked to two killings and his wife's disappearance, came Sunday just before the finale of an HBO documentary about his life in which he said he "killed them all."

Durst was arrested in New Orleans on a murder warrant just before Sunday evening's showing of an HBO serial documentary "The Jinx: The Life and Deaths of Robert Durst" about his links to three killings.

The 71-year-old was asked in the show's finale about similarities in handwriting in a letter he wrote and another linked to one of the killings. Later, filmmakers said, Durst wore his microphone into the bathroom.

What followed was a bizarre rambling in which Durst said, apparently to himself, "There it is. You're caught," and "What the hell did I do? Killed them all, of course."

The show ended, and it wasn't clear whether producers confronted Durst about the secretly recorded words, or what Durst meant by them.

Durst is being held without bail following an initial hearing in New Orleans magistrate court Sunday and is expected to agree to be taken to Los Angeles to face first-degree murder charges, his lawyer Chip Lewis told NBC News.

The heir had been arrested by FBI agents Saturday at the JW Marriott hotel in New Orleans on a Los Angeles County warrant for the murder of a mobster's daughter 15 years ago just before Sunday evening's finale of "The Jinx," by filmmaker Andrew Jarecki.

Durst, the son of a New York City real estate mogul, is accused in the 2000 killing of Susan Berman — his confidant, and the daughter of an associate to Las Vegas mobsters Bugsy Siegel and Meyer Lansky — in West Los Angeles, according to the LAPD.

"As a result of investigative leads and additional evidence that has come to light in the past year, investigators have identified Robert Durst as the person responsible for Ms. Berman's death," the statement said.

Durst has always maintained his innocence in Berman's death. Berman, 55, a writer who became Durst's spokeswoman, was killed at her home near Beverly Hills with a bullet to the back of her head as New York investigators prepared to question her in the unsolved 1982 disappearance of Durst's wife, Kathleen.

One key piece of evidence proffered in "The Jinx" was a letter, obtained by the show's makers, that Durst had sent Berman.

The handwriting on that letter's envelope bore striking similarities to that of an anonymous letter that had alerted police to a "cadaver" in Berman's home, down to the misspelling of "Beverly Hills" as "Beverley Hills" on both, the documentary showed.

It was not made clear in the documentary, nor by the LAPD, whether the arrest was a result of the discovery of the letter. His lawyer Lewis said Durst had provided handwriting samples to the LAPD in the past, and has "no doubt" the arrest was orchestrated in coordination with HBO's broadcast of the final episode.

Durst's estranged and fearful relatives thanked authorities for tracking him down.

"We are relieved and also grateful to everyone who assisted in the arrest of Robert Durst. We hope he will finally be held accountable for all he has done," said his brother, Douglas Durst, in a statement.

Durst was acquitted of a different murder more than a decade ago, after his neighbor Morris Black was killed and his dismembered body parts were found floating in a Galveston, Texas, bay.

Durst admitted that he used a paring knife, two saws and an ax to dismember Black's body before dumping his remains, but said he had killed Black in self-defense.

He was also investigated in the 1982 disappearance of his first wife, Kathleen Durst, who went missing from their cottage in South Salem, New York, in 1982. No one was ever charged.

Durst is the son of the late Seymour Durst, patriarch of the billion-dollar Durst Corporation, which owns and manages several skyscrapers in New York City, including 1 World Trade Center, according to The Associated Press.

Back in December, Durst pleaded guilty to a misdemeanor charge of criminal mischief for exposing himself and urinating on candy at a Texas CVS drugstore.

Durst "has been incredibly lucky that so many people who've investigated him have dropped the ball, but I think that luck may be running out," said former Galveston County District Court Judge Susan Criss, who presided over the Texas murder trial.