Perhaps the EU is trying to make the UK feel a little better about their Brexit vote. The highest court of the EU, the Court of Justice (is there another kind?) released a decision regarding a recent case in which a man blames his multiple sclerosis (MS) of the Hepatitis B vaccine.

The pronouncement was not a decision on this specific case, but general guidance for EU courts on how to evaluate evidence of a possible causal link between a product and alleged harm. I’ll discuss the general principles at stake first, and then the details of this specific case.

The current standard is that anyone claiming damage from a product has the burden or proof to demonstrate that there is a defect in the product and that defect caused harm. The current ruling deals with the threshold for meeting that burden of proof. They write:

“In the present case, the Court considers that the temporal proximity between the administering of a vaccine and the occurrence of a disease, the lack of personal and familial history of that disease, together with the existence of a significant number of reported cases of the disease occurring following such vaccines being administered, appears on the face of it to constitute evidence which, taken together, may lead a national court to consider that a victim has discharged his burden of proof. That could be the case inter alia where that evidence leads the court to consider, first, that the administering of the vaccine is the most plausible explanation for the occurrence of the disease and, second, that the vaccine therefore does not offer the safety that one is entitled to expect.”

So essentially they are saying this:

The plaintiff must still meet their burden of proof

In the absence of definitive scientific evidence of a causal link, the court may make up their own mind about a link based on serious evidence.

Such evidence includes the absence of a family history for the disease, prior healthy state, a temporal association, and other cases with a similar association.

They also state the reason for this decision is that, in the absence of a scientific consensus it may be impossible for an injured party to prove causation. They also claim this does not shift the burden of proof to the defendant, but the decision is already being criticized by experts for doing just that.

With this decision the EU court is lowering the standard of evidence to prove harm from a product. They acknowledge that there must be a balance in which companies are protected from wrong or frivolous claims, but that consumers must also be protected from harmful products. Their goal is to shift this balance in favor of consumers. I don’t want to argue with or debate that goal, because that is mostly a value judgement. We can bring evidence to bear about the consequences of such a shift.

Mainly we can look back at case law and see how often were companies penalized for harm that scientific evidence eventually disproved, vs how often were companies let off the hook for harm they actually caused because the scientific evidence was slow to prove harm.

Another angle here is the shifting of the evaluation of evidence from scientific experts to judges. The ruling essentially says that the plaintiff does not have to produce experts who will say that there is scientific evidence of a link – judges can conclude there is a link by themselves based on the criteria outlined above. This, I think, is the most dangerous part of the ruling.

By comparison, the standard in Federal courts in the US is the Daubert rule:

“Although the language of the majority opinion in Daubert appears to confer broad discretion on trial judges, it emphasizes the need to determine the reliability of scientific knowledge through its origin in the scientific method. To that end, the Supreme Court directed trial judges to consider at least four factors when determining admissibility: (1) whether the theory or technique can be tested, (2) whether the proffered work has been subjected to peer review, (3) whether the rate of error is acceptable, and (4) whether the method at issue enjoys widespread acceptance.”

So Daubert has trial judges as the gatekeepers of scientific experts, rather than substituting their own judgement about the evidence for that of experts.

Hepatitis B Vaccine and MS

Let’s look at the specific case that prompted this ruling, and I think it will become more clear why this is problematic. First, when assessing risk we have to recognize that it is impossible to prove zero risk, because that would take infinite data. A demonstration of an absence of risk is always, therefore, proportional to the amount of evidence currently available. There is no magic threshold in which the essential absence of risk is proven. All we can say is that any potential remaining risk is below a certain amount, determined by the existing evidence.

The court does not make clear in their guidance how much evidence is sufficient to say that the scientific evidence supports the conclusion that there is probably no causal link. How are judges supposed to make that determination, other than listening to the expert consensus?

If there are no studies, that would make the determination easy. That is not the case, however, with hepatitis B and MS. Systematic reviews from 2003, 2011, and 2017 all found no increased risk of MS after any vaccination, specifically including hepatitis B. The 2017 review looked at 62 studies of vaccines and MS risk. Why was this not enough?

The other criteria are also very problematic. In medicine we think of such factors (or at least we should) in terms of their predictive value. How predictive is it that the plaintiff was healthy before they developed MS, in terms of their being a specific trigger of their MS? Not at all. That is how almost everyone develops MS – they suddenly have their first clinical episode and are eventually diagnosed with MS. MS tends to strike young otherwise healthy people. Whether or not a person was otherwise healthy before they developed MS has, in my opinion, zero predictive value in determining the cause of the MS.

For background, MS is an autoimmune disease of the central nervous system. It is still largely mysterious why some people develop MS. Also MS is not inherited, we do know that there are some genetic predispositions. Having a first degree relative with MS increases one’s risk of developing MS, probably because of certain genes that affect the immune system. Epidemiological studies also show that there are high risk regions and that people have the risk of where they lived until they were about 15 years old. This suggests that childhood exposure to certain viruses or bacteria may interact with a susceptible immune system to produce MS.

With this in mind, let’s look at the next two factors – family history and temporal association. About 15% of patients with MS will have a family member with MS, which means that 85% have no family history. So – how predictive is the absence of a family history when addressing the question of a specific alleged trigger? Not very much.

In fact you can flip the logic entirely. It is probable that MS results from the interaction of genetic predisposition and an environmental trigger. If your argument is that a vaccine is the trigger, it is more likely (not less likely) to occur in someone with a family history who has the susceptible genetics. The absence of a family history may, therefore, have a negative effect on the probability of a specific trigger causing MS. Of course, there is not data on this for vaccine in MS because the evidence shows vaccines are not a trigger.

What about the temporal association? It does make general sense that if you are going to claim that A caused B that you should demonstrate A preceded B. OK, fair enough. But in this case the court is claiming that it is plausible that a vaccine that was given one year earlier was the possible cause of the plaintiff’s MS. In medicine the plausibility of trigger A causing disease B not only depend on the sequence, but also the delay. Different diseases have a different latency (or incubation period if we are talking about infections).

Creutzfeldt-Jakob disease (CJD), for example, has a delay from exposure to disease of years, even decades. Some acute infections may have a delay of several days. Also, for shorter delays the temporal association is more predictive, because there is less opportunity for a chance association. For a delay of months or years, the window of chance opportunity is much greater.

So, is a one year delay between vaccine and MS a plausible relationship? Again, we can’t say specifically, because there is no evidence that vaccines cause MS. We do know, however, that the epidemiology shows a 10 year or greater delay between living in a high risk region of the world and developing MS. So from that perspective a one year delay is probably too short. One year is also a long window of chance opportunity, meaning that by chance alone many people will develop MS withing a year of any potential trigger.

This brings us to our final point – the ruling essentially says that judges can base decisions on anecdotal evidence they find compelling. But how do we know if an association is chance or a real cause and effect? The ruling implies that if there are “many” people with the correlation that can be enough. Of course, how many is “many”? You further have to consider the fact that once the notion of a possible disease trigger gets out into the public, this will attract the attention of people with the disease. This triggers a collective confirmation bias, with people coming forward self-selected for having a possible correlation.

What we need to do is look systematically at exposure to the potential trigger and the risk of developing the disease. But of course, that’s doing science. That brings us back to the old standard of asking experts what the scientific evidence says. It also shows the folly of letting non experts (judges) decide if they think there is a correlation based on anecdotal evidence and confirmation bias.

I think you can also see how naive the judgement was in that it reflects a perspective devoid of any specific knowledge of medicine in general and MS specifically. The judgement naively assumes that a prior healthy state, the absence of a family history, and a broadly defined temporal association have some predictive value, when they clearly don’t. Expertise in MS and even just medicine in general reveals all of these assumptions to be unwarranted. That is exactly why experts are needed to interpret the evidence.

Conclusion

While I understand the legal reasoning behind this decision, and the desire to protect consumers who may have been harmed and should not be burdened with meeting an unreasonable demand for scientific evidence – scientifically, this is a horrible decision. This was a decision by non-medical experts downgrading the role of medical experts based upon reasoning that is naive in a way that specifically reflects the absence of medical expertise. This was Dunning-Kruger manifesting in a legal decision at the highest level of the EU.