I understand that fervent skepticism can sometimes be annoying, especially when it is aimed at something you believe or at least think is either likely or should not be ruled out. It’s as if the skeptics are trying to disprove your belief.

Well, they are. That’s the point. Welcome to science.

Initial skepticism is a productive response to any new claim, and history bears that out. Often observations or even experimentation leads to the possibility of new phenomenon. Almost by definition this is based on a currently unexplained anomaly.

An anomaly is a phenomenon that does not fit with our current understanding of the universe. If we are encountering an entirely new phenomenon it is likely that what we are observing will be anomalous, because we can’t explain what we don’t know. In fact scientists love anomalies – they point toward new discoveries. Anomalies are where the action is.

But here is where I find there is often a disconnect between scientists and the public, exacerbated by pseudoscientists: when scientists encounter an anomaly, what does that mean and how do they proceed? I often find that pseudoscientists encounter an anomaly and simply declare the anomaly as evidence for (or even proof of) the phenomenon for which they are looking. “You see that electromagnetic field? That’s a ghost.”

Sometimes the name given to an apparent phenomenon gives away the fact that they are jumping to conclusions. Cold spots in old houses are called, “ghost cold.” The term “extrasensory perception” assumes a certain conclusion. More recent researchers, trying to sound more scientific, have adopted the phrase, “anomalous cognition,” which is better.

The History of Anomalies

Scientists, however, approach anomalies very cautiously. Their initial response is (or at least should be) skepticism, followed by a dedicated attempt to disprove the anomaly, or explain it as an error, a fluke, or as part of known phenomena. If they get too enthusiastic and get ahead of the evidence, their colleagues are likely to slap them down, and they won’t be gentle.

Scientists, of course, are people, and every scientist wants to make that huge discovery which will make them famous, so there is a massive incentive to believe, or at least hope, that the new anomaly you have discovered represents a phenomenon new to science, rather than an error in your experimental setup. That is why the scientific community will reign in premature enthusiasm.

There are plenty of examples of this process playing out – a new apparent anomaly leads to some exciting speculation but the scientific community is skeptical, and eventually careful analysis reveals the anomaly to be an artifact or error.

The faster-than-light neutrinos is a recent event that now stands as an iconic example of why initial skepticism is the proper result. Whenever someone thinks that a new phenomenon was just discovered, you will hear the phrase, “remember the faster-than-light neutrinos. As science magazine reports:

The OPERA team had timed neutrinos fired through Earth from the European particle physics laboratory, CERN, near Geneva, Switzerland, and found that they made the 730-kilometer trip to Gran Sasso 60 nanosecond faster than they would traveling at light speed.

Nothing should be able to travel faster than light. Further, we have detected neutrinos from distant supernova, and never detected they were traveling faster than light. The OPERA team knew this, they knew their findings were almost certainly an artifact, but after a dedicated attempt to find the error they could not, so they presented their results and asked the scientific community to find their error. I’m sure they were wishing that the scientific community would also fail to find the error, and that they had in fact discovered the first evidence of something breaking the ultimate speed-limit, but they were being proper scientists. Other teams around the world tried to replicate their results and couldn’t. Then, within just a couple of years:

But in February, the OPERA team also discovered that a loose fiber optic cable had introduced a delay in their timing system that explained the effect.

Anomaly solved – no faster than light neutrinos.

In 1989 Martin Fleischmann and Stanley Pons announced that they had discovered cold fusion. They were getting a little ahead of themselves – they had detected a tiny amount of anomalous energy in their experimental setup, energy they could not account for. You will notice that 26 years later we are still not powering the world with cold fusion. Labs around the world failed to replicate their results.

That’s the problem with tiny anomalies – they can result from tiny errors. It is really heard to rule out tiny errors. That is basically the entire history of the free-energy community. Tiny anomalous energy readings are used to claim a new phenomenon of some kind of free energy exists, with promises that the process can be scaled up and power the world. They never scale up, however, because the tiny errors don’t scale, or if they did they would become obvious.

Very recently astronomers discovered a star, KIC 8462852, with an anomalous pattern of light fluctuation. This likely means there is something odd in orbit around this star. Nothing currently observed would explain the pattern. This is the phase of a discovery where scientists are free to speculate wildly, as long as they don’t confuse speculation with conclusion. The most exciting speculation is that the pattern is the result of a massive alien construct.

This is also where Occam’s razor cuts in. The speculations, or hypotheses, that introduce the fewest new assumptions are the most likely to be true. We should only resort to the more wild speculations when the mundane ones have been ruled out. The anomalous light patterns are more likely to be comets than aliens.

This doesn’t mean the light patterns cannot be aliens, just that we shouldn’t get ahead of ourselves. No one would be more excited than me if they turn out to be aliens. However, recently SETI turned their radio telescopes toward KIC 8462852 and found nothing. This doesn’t prove no aliens, but the unlikely has just become really unlikely.

Over the last couple of years scientists at NASA and elsewhere have been testing what is called an EM drive. The drive, which is claimed to produce thrust without propellant (hint: that’s a new phenomenon not currently known to science), produces a tiny amount of anomalous thrust in testing. Just a week ago it was announced that the latest round of testing is still producing anomalous thrust.

While such a drive would be an awesome piece of technology, I am not getting excited for the same reason I am not investing in cold fusion. The claims are based upon tiny anomalies, with the speculation that they can be (here are those famous words again) scaled up. Each time a new experiment shows continued anomalous thrust it is reported as showing that “the EM drive works.” However, each experiment is just trying to eliminate one possible source of error, just like the OPERA scientists tried to eliminate each source of error they could think of.

This is why extreme caution is appropriate when dealing with anomalies. It doesn’t matter how many sources of error you eliminate, if there is still one source of error remaining. the problem is, it is impossible to prove a negative, that no source of error exists. You can only eliminate sources of error you can think of and control for.

This is where judgement comes into play – when have we ruled out enough sources of possible error? The best way to think about is this – never. We have never ruled out that what we think is going on might be do to some other unknown process. We simply grant tentative assent to one theory for a possible new phenomenon when it has withstood all “reasonable” challenges. We can then proceed further to discover how the new possible phenomenon works, what are its properties. Eventually we can piece together understanding of the phenomenon, and it may even enter the realm of known science. Even there, acceptance is tentative, even though the probability is approaching one.

In the case of a technology, for example that produced alleged energy or thrust, there is also a simple test. Can the technology be useful? I will accept cold fusion or zero point energy when I can run my house off such a device. When NASA is sending probes to Jupiter using the EM drive, then sure, the technology will be undeniable. As long as such phenomena live in the realm of tiny anomalies, I will remain skeptical.

In areas where the alleged phenomenon is subjective, such definitive testing is rarely possible. This is why alternative medicine pseudoscience thrives – the anomalies that are driving belief in the alleged new phenomena (like life energy, for example) are entirely subjective. It takes really careful rigorous study to determine if the phenomena are real, studies that are rife with error and bias.

All of this is why ultimately it is so important to consider scientific plausibility. How much time and effort should we invest in a possible new phenomenon? That is a judgement call, and part of that judgement is plausibility.

Conclusion

I hope 50 years from now we are powering our civilization with cold fusion, riding in hover cars propelled by some version of the EM drive, and decoding the Encyclopedia Galactica beamed to use by an alien civilization. The current evidence for any of this, however, is not convincing.

Here is the most important lesson, however. If we are going to eventually get there, to the point where we can conclude that a new phenomenon is likely real, we will get there by traversing the gauntlet of fervent skepticism. Scientists ask, “how can I be wrong,” and then design their next experiment. We make progress by trying to prove new ideas wrong. Good scientists are skeptics, and history shows that initial skepticism is not only reasonable, it is necessary.