By Grant Jacobs • 31/12/2016 • 40

‘Substantially equivalent’ is the term used by regulatory bodies as part of confirming a GM crop is safe for consumption. Earlier work claims GM corn is ‘substantially equivalent’ to non-GM corn.

Earlier this month a study was published in Scientific Reports claiming genetically modified corn is not substantially equivalent to non-GM corn, “Our molecular profiling results show that NK603 and its isogenic control are not substantially equivalent.”

Plant biologists have said this research doesn’t show what it claims to.

Rather than repeat what others have already said, I’m going to offer a brief explanation of what they have said.

For those in a hurry the main point that has been made is that they haven’t found what the typical range of amounts of each protein* is first, and without that you can’t tell if the differences they found are unexpected or not—the results end up hanging the air, neither here nor there.

There are other points, too, such as if the main differences observed are due to a fungal infection.

Since this was written a few other pieces offering criticism have appeared:

I’ve excerpted portions of these in the comments below this piece. If there are others, let me know in the comments below.

The authors and the scientific journal

The authors are an international group of scientists, the best known who would be Gilles-Eric Séralini whose previous work has been widely criticised. The first author has collaborated with Séralini for several years (judging from the references cited).

The journal the work is published in is Scientific Reports, not Nature as many are saying. Nature is a very prominent journal. Scientific Reports is a much more modest affair offered by the same large publishing company, Nature Publications Ltd. Scientific Reports is an open-access journal. Like a few other open-access efforts, it accepts papers on technical merit, rather than if the work is especially meritorious or not.

Expert reaction at the UK Science Media Centre

Some expert comments are available at the UK Science Media Centre.

The first comment is by Dr Dan MacLean, Head of Bioinformatics at The Sainsbury Laboratory. The Sainsbury Laboratory is one of the biggest (if not the biggest) plant laboratories in the UK. They have a lot of experience studying plants. Loosely-speaking bioinformatics is the field that works with biological data using computational (mathematical, statistical) methods.**

His first point is the same as I made earlier,

A big issue with this analysis is that materials were collected under potentially quite different conditions. Different parts of the same farm, potentially different chemical makeups in the soil, different water contents, different elevations, exposures and temperatures. Under tight laboratory conditions the metabolome and proteome are very variable and the statistics presented here do not go anywhere near controlling for those factors. “There are a huge amount of things that could be affecting the expression and levels of everything in those plants and no exploratory and controlling statistics are presented. The analysis just jumps straight into ‘everything is equal, let’s do tests’ […]

Dr Joe Perry, former Chair of the European Food Safety Authority GMO Panel picks up on this, too, noting that the EFSA checks,

In contrast with compositional analysis, which is done for every application, and reported by EFSA, and which involves proper replicated field trials, this study appears to have been done with single, unreplicated plots. Therefore it is not possible to say with any certainty whether the differences reported are due to differences between the treatments or differences between the two fields (or two plots within the fields) used. In other words the basic tenets of experimental design seem not to have been followed. For that reason I could not yet describe this as a thorough piece of science.

His last sentence is pretty damning, really. He’s saying the science done has a pretty incompetent blunder in it, that with a little thought, it should have been obvious from before the work was attempted that it couldn‘t show what they wanted to test.

In order to show that a difference is biologically significant, you need to know what the ‘normal’ range of differences are. Is the difference you see within that normal range. Or is it well outside it? To do that you first need to know the normal range.

Even if you accept a difference is a valid difference, you want to know if it’s meaningful. The last expert, Prof. Johnjoe McFadden, Professor of Molecular Genetics at the University of Surrey, picks on that. He seems to have taken the author’s conclusion on faith, but suggests that all these types of studies are fundamentally flawed, that the amounts of proteins in plants differ too easily for them to be meaningful,

“How equivalent does it need to be? If you perform this detailed level of analysis on any perturbation of any organism you will detect this level of change – organisms are extraordinary sensitive and, for example, similar changes are produced when treated with e.g. pesticide or herbicides1 or when attacked by pests2.

I would expect that practically any perturbation to an organism will generate a response that can be detected by these powerful techniques – that is after all what life does. So all it shows is that GM, like pesticides, herbicides, drought, predation or even growing in a different field will produce a response by the organism. If GM was banned on these grounds then so would all herbicide pesticides and indeed anything that causes a change (which is everything).

You could read this as saying that these comparative tests are mostly effort to soothe, to be politically correct, that even if a difference were observed it most often wouldn’t mean much. It’s worth remembering that these comparative tests aren’t meant to be the last word, but rather to find those cases that might want looking into, and to eliminate the rest.

This, to me, brings back to a core point about complaints about GMOs: it’s if something affects something else that is wanted. That’s not about if the plant is a GMO or not. Just what affects they have. It’s why I continue to say that the term ‘GMO’ is a red herring. (And a damn waste of time, too!)

MacLean and Perry both point out that having identified a (major) issue in the original data, they’re not really able to draw conclusions,

This has the effect of making the decisions about what pathways are changing moot. No clear conclusions can be reached, and certainly not on the basis of p-values. Hence all downstream analyses could not be expected to show clearly any patterns because of considerable noise in the list of things that are changing. Further details about the conduct of the experiment would be useful to confirm or otherwise this initial impression.

Comments following the paper

Another source of commentary are the comments following the research paper itself.

In many ways these points are moot given the issue of not having established what the normal variation is. Nonetheless a few interesting questions are asked.

These comments are unfortunately polluted by some uninformed commenters, as well as some sloppy comments by some who, in my opinion, should know better.

It‘s further confounded by that some of the comments have been removed. According to one comment, at one point there were over 130 comments: as I write there are 84.

With that in mind, I’ll select just a few that are more meaningful, each under their own header so that you can skim for the ones that interest you.

Others call out the basic design of the experiment

Paul Vincelli offers,

By my reading of this paper, the three corn plantings that represented the source of grain samples for the three experimental treatments studied were not spatially randomized/replicated in the field. Therefore, the statistical effect of treatment is confounded with the effect of planting position, making it seemingly impossible to statistically separate one effect from the other. Even in such highly controlled environments as growth chambers, plants in different positions can produce significant differences in growth. In the field, position effects are even more likely, as variation in the physical, chemical, and biological environment can be substantial even in sites which appear to be superficially similar. Therefore, the conclusions about the treatment effects reported here are called into question. I know of no statistical analysis that can overcome this design flaw. Others have independently identified this as a major concern, as well (http://www.sciencemediacentre.org/expert-reaction-to-multiomics-analysis-of-nk603-gm-maize/) The paper seems not to discuss -omics effects of conventional plant breeding approaches, which are substantial (Sustainability 2016, 8(5), 495; doi:10.3390/su8050495). For the record, I report no conflicts of interest in the topic of genetically engineered crops (GMOs).

Similarly, Rod Herman writes,

Before deeply considering the importance of the analytical results, I would think one would look at the experimental design of the field experiment from which the grain samples came. Can someone please explain how grain samples from unreplicated field plots can be used to determine anything about the effects of the crop genetics or production method? Am I missing something? Is it now scientifically acceptable to evaluate crop varieties from single unreplicated plots at one location?

The main difference is likely to be a fungal protein

‘Mem_somerville’ has asked if there is fungal contamination in the GM corn,

There’s a lot of nonsense drama below now, but I want to hear from the authors (Robin Mesnage asked me to post here, but I can’t see if he’s responding): 1. What is your explanation for the fact that top fold-change proteins in your data set are fungal proteins (and it’s a known maize pathogen)? 2. Are you aware that fungal contamination could result in similar changes in regards to the pathway changes that you describe? Did you consider this at all? Why didn’t you address this in your paper? 3. If you wish to dismiss your own top reported proteins, how can you stand by the importance of the fold-change claims you are making about other proteins? Thanks for your guidance on this. It’s very perplexing.

According to others, these techniques are able to detect very early stages of infections that not yet visible to the eye. I’m wouldn’t be surprised – these are very sensitive techniques. In fact, in many ways they’re too sensitive. This is a recurring problem in all these system-wide screen using highly sensitive molecular techniques: you end up having to be extremely careful to test if the variations you see reflect what you are testing, not something else as the techniques are able to pick up differences that have been caused by the most mundane reasons.

One issue for these studies is contamination; testing for it needs to be built in. Infection is a type of contamination in a sense – the sample isn’t just the plant. Infections can cause metabolic changes, of course, which would be a possible reason for the differences observed.

One suggestion, by ‘Rightbiotech’ was that the ‘top’ difference might be from a worm, rather than a fungus. If so, it would also be problematic (if not fatal) for this study, as that would be contamination. (Contamination large enough to show up dominantly in the results.)

For the curious, the fungus in question is Gibberella moniliformis, a pathogen of corn.

Does the study test what it claims to?

This one struck me, too.

On reading the abstract I realised the paper set out to test if there was any difference in metabolic activity, not the ‘substantial equivalence’ used by the regulatory bodies that it claimed to want to address.

Related to that there is an inherent problem in these large systemic surveys of ‘gold digging’, finding things that are meaningless if you try too hard.

Chris Preston replied to Damian, writing,

If you look at the paper, the stated intention was to address the issue of substantial equivalence: “In an effort to provide insight into the substantial equivalence classification of a Roundup tolerant NK603 GM maize”. However, the authors do not address substantial equivalence as interpreted by regulatory agencies, instead they address something much closer to what you suggest.

What Damian suggested the aim was is,

whether the inserted allele in NK603 caused a phenotypic difference (at proteome and metabolic levels),

Damian goes on to ask,

isn’t the inactivated complement the correct control, and not the “closest isogenic line”.

What he’s asking is that the correct comparison for seeing if there is a difference caused by carrying a inserted gene, would be to compare with the plant with the inserted gene inactivated, rather than a different strain of the plant, which, because it’s a different strain, will have differences in it’s biochemistry too.

Another approach is to compare a range of corns, and learn what is typical. (And then you still have to consider if the differences are meaningful.)

More comprehensively, Chris Preston goes on to write,

The control used should have been the one that addressed the hypothesis put forward. In the case of substantial equivalence, what regulatory agencies look at is whether there is evidence that the crop has a composition that might be outside the range of what humans are already exposed to in their diet, as that may indicate the need for more testing. Therefore, you will see most substantial equivalence tests address not only the non-transformed isogenic line and/or a null transformant, but also the range of known compositions for that crop. If the authors truly wanted to address substantial equivalence, they should have tested the NK603 maize against a range of maize cultivars common in diets. If the idea was to address whether the specific transformation caused differences, then a null transformant would have been more appropriate. There are likely to be a reasonable number of differences between DKC 2678 and DKC 2675 irrespective of the transgene, so the experimental design used would be unable to address that question adequately. In short, this study was flawed from the beginning as it wasn’t set up to test the hypothesis that the authors claimed they were testing. This is irrespective of the issue with Gibberella moniliformis infection that means the conclusions drawn by the authors are unsafe. The presence of significantly more of these proteins in the non-GM corn, despite the authors running a screen just for the the maize proteome, means that it is impossible to tell whether any differences between the two samples were due to fungal infection or the insertion of the gene. The multiple flaws in this study mean that you really cannot conclude anything from the results.

You’ll see he talks about “the range of known compositions for that crop” — that’s the range of levels of proteins (etc) that was referred to at the top of this article.

Note also that he points out that even if you are looking for differences from adding the new gene, the experimental design isn’t able to address that, “There are likely to be a reasonable number of differences between DKC 2678 and DKC 2675 irrespective of the transgene, so the experimental design used would be unable to address that question adequately.”

At Genetic Literacy

It’s worth reading a blog post at the Genetic Literacy blog post as a companion piece to my own, one that delves into a little more detail.

Kevin Folta, who has long been involved in communicating about genetic modification, particularly of crops, has expressed a few thoughts in the comments –

My favorite part of the paper is that they did NOT detect glyphosate on plants sprayed with glyphosate. However, activists claim to detect it in food. The rest of this paper confirms well that the products are essentially the same. The differences observed are not much more than you’d expect from small environmental variations in plant biology. I would have liked to have seen a comparison within samples from the control group (the isoline). I have a funny feeling you’d see variation there too. Small differences in moisture, etc could account for the differences. On the other hand there could be small collateral changes induced by a transgene. No surprise there. The question is, is there any reason to believe the changes observed in metabolites are problematic? No. Not at all. Other plants make the same polyamine compounds in mountains relative to corn. The title and discussion were completely inappropriate for a scientific journal and should have been revised. But obviously soft reviewers and editor that let it slide.

More to think about

There’s more to think about, but I’ll leave it at this. This is more than enough to start with! In time there might also be comments at the PubMed source of the research paper.

But let me toss in two minor points, worth noting in a different way –

Claims that it’s the first

They claim it’s the first study of this kind. Sort-of-ish but not quite really. This paper also examines the metabolome of the GM and non-GM varieties of corn. It’s been out since earlier this year, so the authors should have been aware of it. They don’t cite this paper in their references.

They will have used different techniques, but essentially all research papers do that.

Some papers try inflate their work with a claim to a ‘first’. It’s a distraction really: better to just focus on the data and what it might mean in my humble opinion. Besides, I think claims to a ‘first’ are best left to editorials.

Not a good title

I’d also quibble, rather strongly, that the title is inappropriate. It might seem nit-picky of me, but this is the sort of detail tougher scientific journal editors insist on. The title reads, “An integrated multi-omics analysis of the NK603 Roundup-tolerant GM maize reveals metabolism disturbances caused by the transformation process”.

The trouble is that last bit.

Firstly “the transformation process”, strictly speaking, are the steps of genetically modifying the plant. They’re not studying that active process, they examine a resulting product of that, seeds. Also ‘disturbances’ is a loaded term. It implies the differences where through something ‘disturbing’ the ‘natural’ situation. At best they should say ‘differences’.

Footnotes

The research paper refers to maize, rather than corn, but corn is the more familiar term in New Zealand.

* I’m using ‘proteins’ as a bit of a short-cut to keep things simpler.

** It also happens to be my field.

Featured image

Kenyans examining insect-resistant transgenic Bt corn. Source: Wikipedia.Genetically Modified Corn— Environmental Benefits and Risks Gewin V PLoS Biology Vol. 1, No. 1, e8 doi:10.1371/journal.pbio.0000008 http://biology.plosjournals.org/perlserv/?request=slideshow&type=figure&doi=10.1371/journal.pbio.0000008&id=39336 Creative Commons Attribution 2.5 Generic license.