A researcher in China claims to have created the world’s first gene-edited babies, but he didn’t publish the results in an academic journal or provide other scientists with access to the babies. The little data he has revealed suggests that his efforts might not have worked — instead, his work may have put the babies’ health at risk.

Scientist He Jiankui allegedly used the gene-editing tool CRISPR/Cas-9 to disable the CCR5 gene in 31 embryos with the goal of making children who were more resistant to HIV. He claims that two of the embryos were implanted, resulting in the female babies “Lulu” and “Nana.” (His work has since been halted by the Chinese government, reports The Associated Press.)

Other genetics experts are still waiting to verify He’s claims, a process that could take two different forms. If other scientists had access to the babies, they could sequence their DNA and independently see if this gene is no longer there. Without the babies, the results could be confirmed if He provided his raw sequencing data of the babies and their parents, along with all of his notes about his methods and results — which he hasn’t done.

“a failure to meet ethical standards”

The limited data He did reveal at a gene editing conference in Hong Kong this week suggests that the experiment did happen, but maybe not in the way that he intended. There are “all kinds of glitches,” and the data shows that the babies weren’t edited very precisely, says Eric Topol, a geneticist at Scripps Research Institute. Similarly, CRISPR expert Gaetan Burgio, a geneticist at Australian National University, says that the slides He provided showed very amateur work. “I can believe that he did it because it’s so bad,” he says.

Everyone has two copies of each gene, one from each parent, but Nana had only one copy of CCR5 edited. Plus, both babies have a combination of cells with an edit and without an edit, which is called mosaicism. This is “absolutely devastating,” says Burgio, because it means that the babies still have the normal gene — so they’re probably not more HIV-resistant after all.

We do know that using CRISPR to edit a gene can have unintended, negative consequences on health in other ways. Earlier this year, Nature Medicine published papers suggesting that editing one part of the DNA using this method could harm other parts of an organism’s genetic code. This is called “off-target effects.” For example, it’s possible to accidentally disrupt a system that protects from eye cancer when using CRISPR to edit something completely different.

In He’s case, the team found one new mutation in Lulu that could have been caused by CRISPR, but he decided it was safe. Still, Burgio says that the sequencing methods they used to check for off-target effects weren’t enough, and it’s possible the babies both have CRISPR-caused mutations that could harm their health. And even if these off-target mutations aren’t harmful, research has shown that a mutation in the CCR5 gene can make people more susceptible to dying from the flu.

For his part, He has defended himself and even claimed that a second pregnancy with a gene-edited baby was underway. Meanwhile, the scientific community worldwide has been fiercely critical of He’s efforts, which seems to have bypassed typical scientific conventions at nearly every step.

The problem was not simply that he did this in the first place. It’s that He didn’t tell his university or almost anyone else about the experiment, and he didn’t go through the typical academic peer-review process to make sure his evidence was corroborated before publicly discussing it. He likely did not obtain full informed consent from his subjects and might have had a business motive, seeing how he is involved in two biotech startups.

The entire debacle shows why it’s so important to have scientific and ethical safeguards, and it has pushed other geneticists to consider how they would handle this kind of research in the future. Earlier today, at the end of the Hong Kong Summit, the organizing committee released a statement concluding that “the scientific understanding and technical requirements for clinical practice remain too uncertain and the risks too great to permit clinical trials of germline editing at this time.” They also announced plans to set up a “rigorous, responsible translational pathway toward such trials.”

The statement also acknowledges that for Nana and Lulu, rigor and responsibility were absent, calling out He’s research for (among other serious flaws) “a failure to meet ethical standards for protecting the welfare of research subjects.”