Jan. 24 Update: A federal judge today sentenced American David Coleman Headley's to 35 years in prison for his role in organizing scouting missions for the 2008 terrorist attack on Mumbai that killed 160 people. Read our extensive coverage of Headley and his role as a U.S. drug informant and operative of the Pakistani terrorist group Lashkar-i-Taiba below.

This story was co-published with PBS FRONTLINE, which will air a film version today. Check local listings. It was originally published on Nov. 22, 2011.

Prologue: Justice Denied

During a meeting overseas last summer, a senior U.S. official and Gen. Ashfaq Parvez Kayani, the chief of Pakistan’s armed forces, discussed a threat that has strained the troubled U.S.-Pakistani relationship since the 2008 Mumbai attacks: the Lashkar-i-Taiba militant group.

The senior U.S. official expressed concern that Zaki-ur-Rehman Lakhvi, a terrorist chief arrested for the brutal attacks in India, was still directing Lashkar operations while in custody, according to a U.S. government memo viewed by ProPublica. Gen. Kayani responded that Pakistan’s spy agency, the Inter-Services Intelligence Directorate (ISI), had told prison authorities to better control Lakhvi’s access to the outside world, the memo says. But Kayani rejected a U.S. request that authorities take away the cell phone Lakhvi was using in jail, according to the memo to Secretary of State Hillary Clinton and the National Security Council.

The meeting was emblematic of the lack of progress three years after Lashkar and the ISI allegedly teamed up to kill 166 people in Mumbai, the most sophisticated and spectacular terror strike since the September 11 attacks. The U.S. government filed unprecedented charges against an ISI officer in the deaths of six Americans. Yet, Pakistani authorities have not arrested him or other accused masterminds. The failure to crack down on the jailed Lakhvi, whose trial has stalled, raises fears of new attacks on India and the West, counterterror officials say.

“Lakhvi is still the military chief of Lashkar,” a U.S. counterterror official said in an interview. “He is in custody but has not been replaced. And he still has access and ability to be the military chief. Don’t assume a Western view of what custody is.”

In the United States, stubborn questions persist about the case’s star witness, David Coleman Headley, a confessed Lashkar operative and ISI spy. The Pakistani-American’s testimony at a trial in Chicago this year revealed the ISI’s role in the Mumbai attacks and a plot against Denmark. It was the strongest public evidence to date of ISI complicity in terrorism.

But the trial shed little light on Headley’s past as a U.S. Drug Enforcement Administration informant and the failure of U.S. agencies to pursue repeated warnings over seven years that could have stopped his lethal odyssey sooner — and perhaps prevented the Mumbai attack.

U.S. officials say Headley simply slipped through the cracks. If that is true, his story is a trail of bureaucratic dysfunction. But if his ties to the U.S. government were more extensive than disclosed — as widely believed in India — an operative may have gone rogue with tragic results. Both scenarios reveal the kind of breakdowns that the government has spent billions to correct since the Sept. 11 attacks.

The Obama administration has not discussed results of an internal review of the case conducted last year, or disclosed whether any officials have been held accountable.

During an interview in Delhi, former Indian Home Secretary G.K. Pillai asserted that U.S. authorities know more about Headley than they have publicly stated. Several senior Indian security officials said they believe that U.S. warnings provided to India before the Mumbai attacks came partly from knowledge of Headley’s activities. They believe he remained a U.S. operative.

“David Coleman Headley, in my opinion, was a double agent,” said Pillai, who served in the top security post until this past summer. “He was working for both the U.S. and for Lashkar and the ISI.”

The CIA, FBI and DEA deny such allegations.

File photos of Mumbai's Taj Mahal Palace Hotel in November, 2004 (bottom) and on November 26, 2008 (top) as fire engulfs the top floor after a shootout with terrorists. (AFP PHOTO / PAL PILLAI (top) | AFP PHOTO/Getty Images/SEBASTIAN D'SOUZA (bottom))

An investigation by ProPublica and FRONTLINE during the past year did not find proof that Headley was working as a U.S. agent at the time of the attacks. But it did reveal new contradictions between the official version of events, Headley’s sworn testimony and detailed accounts of officials and others involved in the case. The reporting also turned up previously undisclosed opportunities for U.S. agencies to identify Headley as a terrorist threat, and new details about already-reported warnings.

U.S. and foreign officials say his role as an informant or ex-informant helped him elude detection as he was training in Pakistani terror camps and traveling back and forth to Mumbai to scout targets. And three counterterror sources say U.S. agencies learned enough about him to glean fragments of intelligence that contributed to the warnings to India about a developing plot against Mumbai.

In contrast, some U.S. officials say spotting a threat is harder than it seems. Glimmers of advance knowledge are part of the landscape of terrorism. In cases such as the Sept. 11 attacks and the 2004 Madrid train bombings, security forces had detected some of the suspects but not their plots.

“I just have to dispel some of these notions,” said Philip Mudd, a former top national security official at the FBI. “We look at a grain of sand and say … 'why couldn’t you put together the whole conspiracy when you saw that grain of sand?' Well, you got to reverse it. Every day coming into a threat brief, you’re not looking at a grain of sand and building a beach. You’re looking at a beach and trying to find a grain of sand.”

New information about the case comes partly from the DEA. After months of silence, DEA officials recently granted an interview with a ProPublica reporter and went over a timeline based on records about their former informant. The DEA officials said Headley’s relationship with the anti-drug agency was more limited than has been widely described.

The DEA officially deactivated Headley as a confidential source on March 27, 2002, according to a senior DEA official. That was weeks after he began training in Lashkar terror camps in Pakistan and six years before the Mumbai attacks. The senior official denied assertions that Headley had worked for the DEA in Pakistan while he trained with Lashkar in 2002 and beyond.

“The DEA did not send David Coleman Headley to Pakistan for the purpose of collecting post-9/11 information on terrorism or drugs,” the senior DEA official said.

The denial adds another version to a murky story. Officials at other U.S. agencies say Headley remained a DEA operative in some capacity until as late as 2005. Headley has testified that he did not stop working for the DEA until September 2002, when he had done two stints in the Lashkar camps.

Some U.S. officials and others involved say the government ended Headley’s probation for a drug conviction three years early in November 200