For indispensable reporting on the coronavirus crisis, the election, and more, subscribe to the Mother Jones Daily newsletter.

This story first appeared on the ProPublica website.

The NYPD is regularly held up as one of the most sophisticated and significant counterterrorism operations in the country. As evidence of the NYPD’s excellence, the department, its allies, and the media have repeatedly said the department has thwarted or helped thwart 14 terrorist plots against New York City since September 11.

In a glowing profile of Commissioner Ray Kelly published in Newsweek last month, for example, journalist Christopher Dickey wrote of the commissioner’s tenure since taking office in 2002: The record “is hard to argue with: at least 14 full-blown terrorist attacks have been prevented or failed on Kelly’s watch.”

The figure has been cited repeatedly in the media, by New York congressmen, and by Kelly himself. The NYPD itself has published the full list, saying terrorists have “attempted to kill New Yorkers in 14 different plots.”

As Mayor Michael Bloomberg said in March: “We have the best police department in the world, and I think they show that every single day and we have stopped 14 attacks since 9/11 fortunately without anybody dying.”

Is it true?

In a word, no.

A review of the list shows a much more complicated reality—that the 14 figure overstates both the number of serious, developed terrorist plots against New York City and exaggerates the NYPD’s role in stopping attacks.

The list includes two and perhaps three clear-cut terrorist plots, including a failed attempt to bomb Times Square by a Pakistani American in 2010 that the NYPD did not stop.

Of the 11 other cases, there are 3 in which government informants played a significant or dominant role (by, for example, providing money and fake bombs to future defendants); 4 cases whose credibility or seriousness has been questioned by law enforcement officials, including episodes in which skeptical federal officials declined to bring charges; and another 4 cases in which an idea for a plot was abandoned or not pursued beyond discussion.

In addition, the NYPD itself does not appear to have played a major role in breaking up most of the alleged plots on the list. In several cases, it played no role at all.

The NYPD did not respond to requests for comment on the list of 14 alleged plots and how it was assembled. Update (7/10): Bloomberg countered our story Tuesday afternoon, saying “we’ll never know” how many plots the NYPD has truly thwarted.

The following is a breakdown of the plots on the NYPD’s list.

Substantially developed or executed plots:

The failed 2010 attempt by Faisal Shahzad to set off a crude car bomb in an SUV in Times Square. Shahzad was in contact with the Pakistani Taliban before and after the attempt, according to the government, and he pleaded guilty to charges stemming from his role in the attempt. The plot was widely seen as a law enforcement failure, as Shahzad was able to plant the rigged car in Times Square without being on the radar of the NYPD and other agencies.

The thwarted 2009 plot by three former high school classmates from Queens to set off bombs in the subway system. One of the plotters, an Afghan immigrant named Najibullah Zazi , testified in court this year that he and the others had received training from “Al Qaeda leaders” in Pakistan. He also admitted bringing bomb-making materials into New York. All three men have pleaded guilty or been convicted of terrorism charges. According to the AP, the plot was uncovered not by the NYPD, but rather by an email intercepted by US intelligence.

Transatlantic plot: In August 2006, British authorities arrested a group of men who were later charged with plotting to blow up planes bound for North America from London with liquid explosives. (This is the plot that led to restrictions on carrying liquids aboard planes.) The plot is included on the NYPD’s list of 14 because, according to British authorities, one of the men had a memory stick that had information on flights bound for several Canadian and American cities, including, in one case, New York. The plan was to blow up the planes over the ocean. During the trial, there were questions about whether the men were going to act on the plan imminently. Three consecutive trials in the case ultimately resulted in eight convictions. The NYPD was not involved in thwarting the plot.

Cases with significant or dominant role by government informants:

The case of the Newburgh Four , in which four men from upstate New York planted what they thought were real bombs outside synagogues in the Bronx. The men were found guilty in the case in 2010 after the jury rejected an entrapment defense. The bombs were fakes supplied by the government. An informant posing as a Pakistani terrorist recruited Walmart employee and Muslim convert James Cromitie over nearly a year, giving him gifts, including rent money and a trip to an Islamic conference. The informant plied Cromitie with offers of $250,000, a luxury car, and a barbershop. An FBI agent on the case acknowledged under cross-examination during the trial that the government was essentially in control of what the four were doing while they were with the informant. The government maintained that Cromitie was an anti-Semite who talked about committing acts of violence and posed a real threat. A judge who rejected an appeal last year nevertheless called the government’s conduct in the case “decidedly troubling.”

Herald Square: Pakistani immigrant Shahawar Matin Siraj was arrested in 2004 and convicted in 2006 at the age of 23 of conspiracy to bomb the Herald Square subway station in Manhattan. The jury rejected an entrapment defense. An informant for the NYPD’s Intelligence Division played a key role in the case and was paid $100,000 by the government over a roughly three-year period. He told Siraj he was a member of a (made-up) group called “the Brotherhood” that would support a bomb plot. Siraj was recorded talking to the informant about blowing up bridges and other places in New York City, including the Herald Square subway station. The informant later told Siraj that the Brotherhood had approved the plot and that a leader of the group was “very happy, very, very impressed” with the idea. The informant told Siraj the group wanted him to put backpack bombs in the station, and he drove Siraj and another man to the station to do surveillance. At his sentencing, Siraj apologized to the judge but maintained he had been “manipulated” by the NYPD informant. Siraj did not obtain explosives, there was no timetable for the plot, and there was no link to any foreign terrorist group, according to the New York Times.

JFK Airport: Russell Defreitas, a naturalized American citizen from Guyana and former airport cargo handler, and Abdul Kadir, of Guyana, were arrested in 2007 and convicted in 2010 of conspiring to blow up fuel tanks at JFK airport. At the press conference announcing the charges, a federal prosecutor said the public was never at risk. A law enforcement official described Defreitas, 63 at the time of his arrest, to the Times as “a sad sack” and “not a Grade A terrorist.” Pipeline experts told the paper that the men’s plan to blow up a wide area was not feasible. Defreitas was recorded making odd comments talking to the informant, saying he wanted the attack to be “ninja-style” and that the airport was a good target because “[t]hey love JFK — he’s like the man. If you hit that, the whole country will be in mourning.” An informant on the case was a convicted drug dealer paid by the government and worked in exchange for a lighter sentence in a pending drug case. He drove Defreitas to the airport several times to do surveillance with a camera that the informant had purchased for Defreitas. The informant also provided plane tickets to South America and, with the help of the FBI, secured a New York City Housing Authority apartment for Defreitas (that was under surveillance).

US officials (often anonymous) question credibility or seriousness of cases:

The case of Jose Pimentel , a Manhattan Muslim convert who was arrested in November 2011 on “rarely used state-level terrorism charges” after federal authorities took a pass on the case. Manhattan District Attorney Cy Vance Jr. alleged that Pimentel had been “building pipe bombs to be used against our citizens.” City authorities said Pimentel had no contacts with foreign terrorist groups and called him a “lone wolf.” In a series of leaks, federal authorities expressed skepticism that Pimentel was a threat. A federal source told the New York Post that Pimentel was a “‘stoner’ who wasn’t a real danger to anybody other than himself.” Another source cited by the Post questioned his mental faculties, saying he had once tried to circumcise himself. A federal source told DNAinfo criminal justice editor Murray Weiss: “Let’s just say there were issues whether [Pimentel] had the ability to do this without the intercession of the confidential informant.” Weiss also noted that the informant went shopping with Pimentel at Home Depot to buy pipe bomb materials, and the bomb was constructed at the informant’s apartment. Pimentel, who reportedly could not afford to even pay his cellphone bill, has pleaded not guilty.

The case of Ahmed Ferhani and Mohamed Mamdouh , alleged “Islamic extremists” and “lone wolves” who lived in Queens and were arrested in May 2011 after buying guns, ammo, and an inert grenade from an undercover police agent. The authorities alleged that the two men were planning to attack a New York synagogue because they were upset with how Muslims were being treated around the world. But the case was another example of an alleged plot that the FBI took a pass on. Citing federal law enforcement sources, WNYC reported that the FBI passed on the case because the bureau found the undercover operation “problematic” and the allegations “overhyped.” And the website NYPD Confidential reported that Ferhani has a history of mental illness. The AP noted the case faltered early on when a grand jury declined to indict the two men on a high-level terrorism conspiracy charge. Both have pleaded not guilty to lesser terrorism charges.

Brooklyn Bridge: Ohio truck driver Iyman Faris pleaded guilty in 2003 to providing material support to Al Qaeda; he met with senior Al Qaeda leaders abroad and researched how to attack the Brooklyn Bridge. Terrorism analyst Peter Bergen has called Faris “an actual Al Qaeda foot soldier living in the United States who had the serious intention to wreak havoc in America” but “not much of a competent terrorist.” Bergen wrote the plan Faris researched to sever the bridge’s cables with a blowtorch as “akin to demolishing the Empire State Building with a firecracker.” The plot never got off the ground. According to the Justice Department, Faris sent messages to Pakistan “indicating he had been unsuccessful in his attempts to obtain the necessary equipment.” According to the DOJ, Faris also concluded after a trip to New York that the idea “was very unlikely to succeed because of the bridge’s security and structure.”

PATH Train: In 2006, 31-year-old Assem Hammoud was arrested in Lebanon. FBI officials announced that he and others—who had never met but communicated on the internet—had been plotting a suicide attack on subway tubes under the Hudson River. The Washington Post quoted US counterterrorism officials questioning the credibility of the plot, reporting that they “discounted the ability of the conspirators to carry out an attack.” One official described the matter as “jihadi bravado,” adding, “somebody talks about tunnels, it lights people up.” A counterterrorism official told the Times, “[t]hese are bad guys in Canada and a bad guy in Lebanon talking, but it never advanced beyond that.” Hammoud never visited the United States, and there were no charges brought against him here. In 2012, Hammoud was sentenced to two years in prison in Lebanon, which he had already served.

Cases in which an alleged idea for an attack was not pursued: