On Thursday, Dec. 2, Rosie Redfield sat down to read a new paper called “A Bacterium That Can Grow by Using Arsenic Instead of Phosphorus.” Despite its innocuous title, the paper had great ambitions. Every living thing that scientists have ever studied uses phosphorus to build the backbone of its DNA. In the new paper, NASA-funded scientists described a microbe that could use arsenic instead. If the authors of the paper were right, we would have to expand our notions of what forms life can take.

Redfield, a microbiology professor at the University of British Columbia, had been hearing rumors about the papers for days beforehand. On Monday, NASA released a Sphinxlike press release: “NASA will hold a news conference at 2 p.m. EST on Thursday, Dec. 2, to discuss an astrobiology finding that will impact the search for evidence of extraterrestrial life.” Like a virulent strain of bacteria, speculation exploded over the next three days. “Did NASA Discover Life on One of Saturn’s Moons?” asked Gawker, a Web site that does not often ask questions about astrobiology.

The truth was revealed on Thursday. At NASA’s press conference, the scientists described their research, which was just then being posted on the Web site of the journal Science. They had not found life on one of Saturn’s moons; instead, they had gone to the arsenic-laced waters of Mono Lake in California and isolated a strain of bacteria they dubbed GFAJ-1.

Back at the lab, they grew the bacteria in a broth of nutrients. When they gradually reduced the supply of phosphate (a molecule composed of one phosphorus atom and four oxygen atoms) and replaced it with arsenate (one arsenic and four oxygen atoms), the bacteria still managed to grow. The scientists examined the DNA of these hardy microorganisms and inferred that it contained arsenic.

As soon as Redfield started to read the paper, she was shocked. “I was outraged at how bad the science was,” she told me.

Redfield blogged a scathing attack on Saturday. Over the weekend, a few other scientists took to the Internet as well. Was this merely a case of a few isolated cranks? To find out, I reached out to a dozen experts on Monday. Almost unanimously, they think the NASA scientists have failed to make their case. “It would be really cool if such a bug existed,” said San Diego State University’s Forest Rohwer, a microbiologist who looks for new species of bacteria and viruses in coral reefs. But, he added, “none of the arguments are very convincing on their own.” That was about as positive as the critics could get. “This paper should not have been published,” said Shelley Copley of the University of Colorado.

None of the scientists I spoke to ruled out the possibility that such weird bacteria might exist. Indeed, some of them were co-authors of a 2007 report for the National Academies of Sciences on alien life that called for research into, among other things, arsenic-based biology. But almost to a person, they felt that the NASA team had failed to take some basic precautions to avoid misleading results.

When the NASA scientists took the DNA out of the bacteria, for example, they ought to have taken extra steps to wash away any other kinds of molecules. Without these precautions, arsenic could have simply glommed to the DNA, like gum on a shoe. “It is pretty trivial to do a much better job,” said Rohwer.

In fact, says Harvard microbiologist Alex Bradley, the NASA scientists unknowingly demonstrated the flaws in their own experiment. They immersed the DNA in water as they analyzed it, he points out. Arsenic compounds fall apart quickly in water, so if it really was in the microbe’s genes, it should have broken into fragments, Bradley wrote Sunday in a guest post on the blog We, Beasties. But the DNA remained in large chunks—presumably because it was made of durable phosphate. Bradley got his Ph.D. under MIT professor Roger Summons, who co-authored the 2007 weird-life report. Summons backs his former student’s critique.

But how could the bacteria be using phosphate when they weren’t getting any in the lab? That was the point of the experiment, after all. It turns out the NASA scientists were feeding the bacteria salts which they freely admit were contaminated with a tiny amount of phosphate. It’s possible, the critics argue, that the bacteria eked out a living on that scarce supply. As Bradley notes, the Sargasso Sea supports plenty of microbes while containing 300 times less phosphate than was present in the lab cultures.

“Low levels of phosphate in growth media, naive investigators and bad reviewers are the stories here,” said Norman Pace of the University of Colorado, a pioneer of identifying exotic microbes by analyzing their DNA, who was another co-author on the weird-life report.

I asked two of the authors of the study if they wanted to respond to the criticism of their paper. Both politely declined by email.

“We cannot indiscriminately wade into a media forum for debate at this time,” declared senior author Ronald Oremland of the U.S. Geological Survey. “If we are wrong, then other scientists should be motivated to reproduce our findings. If we are right (and I am strongly convinced that we are) our competitors will agree and help to advance our understanding of this phenomenon. I am eager for them to do so.”

“Any discourse will have to be peer-reviewed in the same manner as our paper was, and go through a vetting process so that all discussion is properly moderated,” wrote Felisa Wolfe-Simon of the NASA Astrobiology Institute. “The items you are presenting do not represent the proper way to engage in a scientific discourse and we will not respond in this manner.”

While Redfield considers Wolfe-Simon’s research “flim-flam,” she thinks it’s fine for the NASA scientists to hold off responding to their critics. She is working on a formal letter to Science detailing her objections. But Jonathan Eisen of UC-Davis doesn’t let the scientists off so easily. “If they say they will not address the responses except in journals, that is absurd,” he said. “They carried out science by press release and press conference. Whether they were right or not in their claims, they are now hypocritical if they say that the only response should be in the scientific literature.”

Some scientists are left wondering why NASA made such a big deal over a paper with so many flaws. “I suspect that NASA may be so desperate for a positive story that they didn’t look for any serious advice from DNA or even microbiology people,” says John Roth of UC-Davis. The experience reminded some of another press conference NASA held in 1996. Scientists unveiled a meteorite from Mars in which they said there were microscopic fossils. A number of critics condemned the report (also published in Science) for making claims it couldn’t back up. And today many scientists think that all of the alleged signs of life in the rocks could have just as easily been made on a lifeless planet.

The controversy over the Martian meteorite still sputters on today because they contain only a few alleged fossils, rather than living bacteria. There are only a limited number of tests that scientists can run on the rocks, and their results remain murky. Fortunately, that’s not the case for GFAJ-1. Critics say that a few straightforward tests on the bacteria would show whether they really do have arsenic-based DNA once and for all. And the NASA scientists say they’re ready to hand out GFAJ-1 to researchers who want to study it. This controversy may be burning brightly at the moment, but it probably won’t burn for long.

Like Slate on Facebook. Follow us on Twitter.