By Grant Jacobs • 21/01/2016 • 34

On January 19th, the NZ Green Party pesticide policy for streets and city parks was announced. They’re touting it as evidence-based policy, pointing at a report commissioned by Steffan Browning.

It’s not.

The report is basically a bought opinion piece, that Steffan Browning knows ahead of time will agree with his views because he’s asked it to.

Extraordinarily (or extraordinarily stupidly), the NZ Green Party are presenting this as evidence-based policy.

I’ve previously written on the need for evidence-based policy; some are linked at the end of this post. I think it’s a principle most people can agree with but the ‘report’ the NZ Greens base this on is not evidence-based policy.

Similarly, I think everyone can agree that we’d like our streets and parks to be safe for our kids, but this report is unhelpful.

A common complaint in response to this announcement on the Green Party Facebook page was that Green supporters are tired of ‘evidence-based’ policy that is not.

What Steffan Browning asked for

Important in reports is how they are commissioned and how they are executed.

The report has Steffan Browning — he of homeopathy-can-treat-Ebola fame — asking someone who openly advocates for a position he likes to prepare something that supports his pre-held position — ignoring what does not.

Here’s what he calls for,

I have commissioned this report to bring together some of the peer-reviewed evidence available to support the call for glyphosate based herbicides to be removed from our immediate environment – reducing exposure in our communities, especially the vulnerable including our children.

Note he asks only for what might shore up his a priori conclusion. This is not what evidence-based policy is about.

Here is the Green Party of Aotearoa determined to say it’s evidence-based.

Evidence-based policy should start with defined questions to be addressed, obtain all evidence related to the questions, filter this evidence for relevance and quality, and finally determine what conclusions can be drawn. It needs to be done by someone with sufficient expertise to aweigh the evidence, and to assess the soundness and suitability of the research.

These have not been done.

The consultant is not a toxicologist

I hate addressing the people in these things, but it must be noted that the consultant is not a toxicologist. Her only qualification appears to be a BSc in Agribusiness.

Her only work experience that might be relevant appears to be her time as a member of a community group, the (overly grandly named) Toxic Agrichemical Advisory Forum. As far as I can make out, this is not a formal working party with specialist skills, as the name might imply, but a group of ‘interested persons’.

In addition to lacking the expertise to review the material, the consultant holds a biased view on the subject. The question here is not that she hold views, but that she holds them in ideological way.

The consultant is an advocate, as readily evidenced by her websites and twitter stream. She is also a contributor to GE Free Bay of Plenty –

Evidence is not related to the question at hand

Most of the material presented is a grab-bag, with little or no effort to relate it to the question at hand.

Unsurprisingly this means much of the material is out-of-context or irrelevant.

For example, the report points to the possibility of glyphosate causing cancer. Following this, the NZ Green Party promotional material writes, “a substance that is probably carcinogenic” – but that is misleading on it’s own, as has widely been noted.

The group they refer to, the International Agency for Research on Cancer (IARC), only presents the strength of evidence for some level of risk for cancer, in some settings. The risk might be quite small or large. The setting might be quite specific, or general. You have to look at the specifics.

There is a widespread problem of the reports from IARC being incorrectly presented, and some of the blame must be laid with the how IARC describe their categories.

The Greens’ statement paints with a broad brush an implied position. Dr Kerry Harrington has related on NZ radio, the risk, if any, is likely to be to those doing the spraying day-in, day-out — that inspection of the specifics suggests the risk is likely not to the general public or the setting in question.

That’s basic stuff to omit. Of course, part of the problem here is what has been commissioned. Browning did not commission a review of the evidence, but a presentation supporting the position he wanted to advocate.

Sifting for quality

No effort has been made in the report to determine what studies are meaningful, or of sufficient standard to include. The effect is it includes studies that are widely criticised as poor.

It’s not that you can find something that supports you view, but that after checking it, it does. This includes eliminating poor science.

On page 12 of the report Bruning concludes a discussion of ‘The State of Science in 2015’ with,

It is the twenty first century, and regulatory dismissal of academic studies is not in accord with good and appropriate scientific practice.

In practice, you should exclude studies that are of poor quality, unable to draw conclusions, or that on closer inspection are not addressing the specific questions the report aims to address. Not doing this is a sign of a lack of quality control.

This is part of what makes formal reviews challenging. They do need to examine the details and work through them and determine what is useful, and what is not. They are more than a list of references that on first appearance seem to support a particular viewpoint. (See also my next section.)

Not quoting sources of science summaries

At several points, a list of research papers (supposedly) supporting a claim are listed. Before each reference is emboldened text offering what reads as a Bruning offering a summary or key factoid believed to be of interest in the reference.

Checking a (small) sample of these statements, many look to be lightly edited versions of statements from various sources (including “anti-” websites rather than research sources).

At kindest this is sloppy or unprofessional. At worst it can be read as plagiarism.

If other sources are to be drawn from they ought to cited (even if they are paraphrased). Material that is substantially the same is best quoted with edits to remove any doubt.

This also suggests Bruning has not proceeded from a formal literature review, but from a open search for material that ‘fits’ the intended conclusion. (e.g. the example below clearly starts from the advocacy website, not the research literature.)

Bruning might have, for example, explained at the head of each list that she was to now proceed searching for what might support her claim from on-line material and their associated research publications, citing each. People would (correctly) point out that this is ‘evidence’ by google search, but at least it would be up-front.

A deeper issue is that this may reflect Bruning lacking of expertise to critique the research and draw out relevant findings, and is relying on other’s words.

This reduces much of the report to a list of “likely sounding things”, stuff found on the internet that “look like” they might support Browning’s cause.

As just one example Bruning’s report writes,

Roundup, at concentrations lower than presently used in agricultural applications, are capable of destroying food organisms widely used as starters in traditional and industrial dairy technologies, such as Geotrichum candidum, Lactococcus lactis subsp. cremoris and Lactobacillus delbrueckii subsp. Bulgaricus.

Activist Post has (identical words emphasised),

… that glyphosate formulations, at concentrations lower than presently used in agricultural applications, are capable of destroying food organisms widely used as starters in traditional and industrial dairy technologies, such as Geotrichum candidum, Lactococcus lactis subsp. cremoris and Lactobacillus delbrueckii subsp. Bulgaricus.[iii]

The grammarly.com plagiarism check reported “Significant plagiarism was detected”, as is readily seen by eye in this case. (Note also widening RoundUp, one formulation, to ‘glyphosate formulations’.)

There are other examples. I have not done an exhaustive study on this. The larger portions of text I checked look to be “borrowed” in this fashion. There are relatively few larger paragraphs; I have not tried many of the smaller paragraphs, as they are harder to verify, but it seems likely that a cut-paste-edit approach has been often used.

If readers choose to read this as plagiarism this would be a serious issue: passing off other’s words as your own is regarded as a form of fraud. I’ll choose to interpret with the benefit of doubt, but not quoting or citing the use of other sources is not suitable practice for a formal report intended to base political party policy on.

Existing reviews not used

The choice of literature for Bruning’s report is deliberately from one viewpoint. This means it leaves out useful reviews. As just one example from last year,

Crit Rev Toxicol. 2015 Mar;45(3):185-208. doi: 10.3109/10408444.2014.1003423. Epub 2015 Feb 26.

Evaluation of carcinogenic potential of the herbicide glyphosate, drawing on tumor incidence data from fourteen chronic/carcinogenicity rodent studies.

Greim H, Saltmiras D, Mostert V, Strupp C.

… There was no evidence of a carcinogenic effect related to glyphosate treatment. The lack of a plausible mechanism, along with published epidemiology studies, which fail to demonstrate clear, statistically significant, unbiased and non-confounded associations between glyphosate and cancer of any single etiology, and a compelling weight of evidence, support the conclusion that glyphosate does not present concern with respect to carcinogenic potential in humans.

The full abstract of this review can be found in the Footnotes.

Footnotes

IARC is the same regulatory group whose report on meat and cancer was widely misreported.

Full abstract of Greim et al.,

Glyphosate, an herbicidal derivative of the amino acid glycine, was introduced to agriculture in the 1970s. Glyphosate targets and blocks a plant metabolic pathway not found in animals, the shikimate pathway, required for the synthesis of aromatic amino acids in plants. After almost forty years of commercial use, and multiple regulatory approvals including toxicology evaluations, literature reviews, and numerous human health risk assessments, the clear and consistent conclusions are that glyphosate is of low toxicological concern, and no concerns exist with respect to glyphosate use and cancer in humans. This manuscript discusses the basis for these conclusions. Most toxicological studies informing regulatory evaluations are of commercial interest and are proprietary in nature. Given the widespread attention to this molecule, the authors gained access to carcinogenicity data submitted to regulatory agencies and present overviews of each study, followed by a weight of evidence evaluation of tumor incidence data. Fourteen carcinogenicity studies (nine rat and five mouse) are evaluated for their individual reliability, and select neoplasms are identified for further evaluation across the data base. The original tumor incidence data from study reports are presented in the online data supplement. There was no evidence of a carcinogenic effect related to glyphosate treatment. The lack of a plausible mechanism, along with published epidemiology studies, which fail to demonstrate clear, statistically significant, unbiased and non-confounded associations between glyphosate and cancer of any single etiology, and a compelling weight of evidence, support the conclusion that glyphosate does not present concern with respect to carcinogenic potential in humans.

The full review is pay-walled. I have no read the full manuscript: my point here is that it was available but not used in the report.

Other articles on Code for life:

Putting government policy on trial

Gluckman on science in small countries

What do scientists want from politicians?

When is a scientific paper political campaigning?

Green Party GM policy and discussion about GE or GMOs