Skepticism is a discipline that aims to help one think clearly in the context of our imperfect minds. Skepticism holds that we have cognitive biases that compromise our ability to interpret reality accurately—the same biases that we recognize in others. To think clearly in the context of our biases, we must learn not to place too much trust in our opinions and intuition, since they may be distorted.

In the context of an amateur study of science, there are several biases that we’ll have to consider:

Cherrypicking —People have a tendency to focus on studies, articles and opinions that support their views, and dismiss those that disagree with them. In many cases, evidence will appear to be conflicting, and the tendency is to weigh those that align with our opinions more heavily.

—People have a tendency to focus on studies, articles and opinions that support their views, and dismiss those that disagree with them. In many cases, evidence will appear to be conflicting, and the tendency is to weigh those that align with our opinions more heavily. Focusing on outliers —Anomalous results tend to be overrepresented in the discussion of controversial topics, since they create a more interesting story. This is especially true in the popular press.

—Anomalous results tend to be overrepresented in the discussion of controversial topics, since they create a more interesting story. This is especially true in the popular press. Lack of expertise—In general, we will be exploring topics outside of our areas of expertise. We’ll have to be cautious, since we will have a limited ability to evaluate the strength of studies.

Given all of this, I’ll suggest some strategies for navigating this minefield in a cautious, responsible way.

Give great weight to expert consensus —The best resources we have for refereeing scientific disputes on controversial topics are scientific consensus organizations. These groups are established to forge expert consensus opinions on a given topic. Examples include the UN Environment Programme, UN Food and Agriculture Organization, American Heart Association, American Cancer Society, Institute of Medicine, as well as government organizations. It is possible to disagree with scientific consensus in a responsible way, but it must be done very carefully, especially as an amateur. Differing from expert consensus on more than a few topics usually indicates a bias.

—The best resources we have for refereeing scientific disputes on controversial topics are scientific consensus organizations. These groups are established to forge expert consensus opinions on a given topic. Examples include the UN Environment Programme, UN Food and Agriculture Organization, American Heart Association, American Cancer Society, Institute of Medicine, as well as government organizations. It is possible to disagree with scientific consensus in a responsible way, but it must be done very carefully, especially as an amateur. Differing from expert consensus on more than a few topics usually indicates a bias. Avoid drawing conclusions from primary sources —Primary sources (scientific studies) are exciting and interesting to read. However we lack the context to critically assess studies outside of our area of expertise. Meta-analyses and reviews attempt to control for weaknesses of individual studies, however they provide a lot of room for framing a hypothesis or deceptive uses of statistics or reasoning. It is much better to let the science shake out in an expert community, and wait for consensus bodies to weigh in.

—Primary sources (scientific studies) are exciting and interesting to read. However we lack the context to critically assess studies outside of our area of expertise. Meta-analyses and reviews attempt to control for weaknesses of individual studies, however they provide a lot of room for framing a hypothesis or deceptive uses of statistics or reasoning. It is much better to let the science shake out in an expert community, and wait for consensus bodies to weigh in. Avoid the popular press entirely —The popular press is usually approaching science from an amateur perspective, and distortions are rampant. The popular media focuses disproportionately on outlier theories, and fail to characterize expert consensus. Approach any new topic as if you’d never heard about it before.

—The popular press is usually approaching science from an amateur perspective, and distortions are rampant. The popular media focuses disproportionately on outlier theories, and fail to characterize expert consensus. Approach any new topic as if you’d never heard about it before. Attempt to research dispassionately —We should all have things that we are passionate about. This should not dictate what evidence we are able to consider. A useful metric is to consider that a claim that you have yet to investigate could be either true or false. Imagine that that you researched exhaustively and found that it is true. How would you respond? Which of your opinions might you have to revise or question? Then do the same, imagining that the claim is false. Usually one will be much easier to accept; this tells you something about your personal bias. However the exercise of considering both is a useful way to keep from being too attached to an outcome. You’ll find that most claims don’t threaten your core beliefs, and in either case you’d adapt and be fine.

—We should all have things that we are passionate about. This should not dictate what evidence we are able to consider. A useful metric is to consider that a claim that you have yet to investigate could be either true or false. Imagine that that you researched exhaustively and found that it is true. How would you respond? Which of your opinions might you have to revise or question? Then do the same, imagining that the claim is false. Usually one will be much easier to accept; this tells you something about your personal bias. However the exercise of considering both is a useful way to keep from being too attached to an outcome. You’ll find that most claims don’t threaten your core beliefs, and in either case you’d adapt and be fine. Look for warning signs—When approaching an apparent controversy, take note of signs of cognitive distortion and bias. Warning signs include: paranoid ideology and conspiracy theories, disparaging mainstream science, shallow populism (eg. talking disparragingly about big pharma, the government, shills, or corporations), researchers without appropriate expertise (eg. computer scientists publishing papers on epidemiology), and journals without adequate peer review (eg. Med Hypotheses).

What isn’t Skepticism?

There are some popular misconceptions about skepticism that I’d like to address briefly.

Skepticism does not mean a default position of disbelief. This can lead to a conspiratorial mindset, where the motives of scientists, governments, and expert panels are called into question. Rather, it means applying a degree of scrutiny commensurate with the probability of the claim. If a claim has widespread support within a discipline, it is usually appropriate to accept that claim.

Similarly, skepticism does not mean believing only what can personally confirm. This sounds rigorous, but is extremely limiting. You only have the resources to become an expert in one, or perhaps a handful of fields. The most reasonable default approach is to contribute to building consensus in your own field, and weighing the opinion of expert consensus heavily in other fields.

Skepticism does not (necessarily) have anything to do with religion. The issue of atheism is important to many skeptics. Others feel that arguing about religion is not a productive use of time. You can be a skeptic even if you think that arguments about religion are tedious.

Skepticism is about asking questions rather than having answers. Sometimes skeptic communities organize around a single issue. A mythology emerges where the skeptic community has the Truth, and everyone who differs are assumedly cavemen. However earnest the inquiry that lead us to these conclusions, this is no longer skepticism, and can develop into a petty dogma.

Skepticism is about acknowledging our own limitations, and working with them to the best of our ability. We can’t do this if we adopt a position of arrogance.

Skepticism and Privilege

Skeptic communities are often disproportionately white and male, and often have a difficult time confronting issues of privilege. This has made me quite reluctant to identify with this community, and deserves some discussion.

I’ve witnessed people who identify as skeptics dismissing accounts of racism and sexism, claiming that they are too evidence-oriented to accept these accounts uncritically. Sometimes these claims are so basic, they would not merit a citation in almost any context (eg. many more women than men experience sexual assault), and yet doubt can persist in the face of overwhelming evidence.

There are a few ways that skepticism can contribute to this. Some may believe that responding with doubt is always more skeptical than accepting plausible claims. Others may believe that skepticism requires doubting anything that they cannot verify first-hand, preventing white males from accepting sexism or racism that they do not experience.

These are examples of bias that seem rather common in skeptic communities. Applying unreasonable doubt to consensus claims is an expression of bias, as is applying a higher degree of scrutiny to members of a particular community.

Skepticism asks us to compensate for our own cognitive biases, and to assume that we share the same cognitive biases that we observe in others. Racism and sexism are very common cognitive biases. Another common bias is the “just-world hypothesis”, a term used to describe the tendency to dismiss accounts of injustice in order to preserve a sense of fairness and personal safety. The reality is that coming to terms with injustice is difficult, and can be frightening.

We should treat consensus from other communities with similar respect as we treat consensus from expert communities. We can defer to data when it is available, but we must not dismiss consensus claims.

My peace with the skeptical community is an uneasy one, but for the moment I’m trying the reform-from-within approach.

And that’s that

The theory of skepticism is really quite simple. Applying it is more challenging.