Quick Tour: For those with limited reading time, the Introduction and

Science Myths sections provide a good overview of this article.

Introduction

In this article I'll describe some common science myths, then I'll try to explain why and how science achieves what it does.

Included in our equation, almost as an afterthought, is a small term that represents science as it really is — a doubt factory . It remains to be seen whether this term can influence the solution of the equation of which it is a part.

On the left-hand side of the science equation, along with the earnest seekers we have people who want to believe in something, who feel betrayed by religion and other fixed belief systems and who hope science will offer a certainty that religion cannot. We also have people who just want to find out things, who want to be entertained, or who seek validation that life makes sense. On the right-hand side of the equation are those who would like to exploit public ignorance of science to sell you something, or who want to entertain you by pretending that science is something it's not. Yet another term in the equation are those who try to shape public policy in irrational ways based on a grave misunderstanding of science itself.

There are others who want science's prestige without the perspiration, who think science means lab coats, clipboards and slickly printed journals. Then there are companies and individuals who encourage public ignorance of science, because this can be turned into a marketing strategy (by promoting the idea that science is truth). Ironically, the least effective science boosters are scientists , who for reasons of personal integrity and professional discipline don't normally get involved in science advocacy, because that stand might appear inconsistent with scientific principles.

Science confusion isn't limited to psychologists. In a recent online conversation about science, a correspondent said, "Michio Kaku is a wildly popular and incredibly intelligent theoretical physicist. He believes in multiverses. I'm going to share his beliefs." When I read this, I was forced to realize there are people who turn to science, not because they understand it, but because they see it as a belief system much like religion, but more respectable and up-to-date.

During my multi-year campaign to try to get psychologists to see the benefit of scientific standards, I've had many interesting conversations about the nature of science itself. During these exchanges it has slowly come to me that most people don't understand science, including many who assert an attachment to science for personal or professional reasons.

Science Myths

Science can only accomplish what it does by encouraging doubt and skepticism, by challenging received wisdom, by being irreverent and subversive. All these traits, essential to science's effectiveness, place it at odds with the religious outlook.

In the context of science, here are working definitions for the terms speculation, hypothesis and theory :

As science becomes more important in our lives, as it acquires and deserves more attention, the number of science myths increases in step. Here are some of them:

Experiment and Theory

To restate a point made above, a scientific theory is an idea that is supported by evidence and is falsifiable in practical tests. If an idea has no evidence but is consistent with existing theories, it is a hypothesis. If an idea is not consistent with existing theories and has no evidence, it is speculation. All these categories have appropriate roles in science, but all scientific ideas must eventually be supported by evidence.

Word Definitions: Language is not science and people arrive at word definitions by consensus. Dictionaries are not meant to tell people how to use words, but to describe how people use words. For example, let's look at the definition of "literally": in a literal sense or manner : actually <took the remark literally> <was literally insane> in effect : virtually <will literally turn the world upside down to combat cruelty or injustice — Norman Cousins> Definition (2) flatly contradicts definition (1). The reason? When words are used in contradictory ways, a dictionary can only report this fact — so it does. Words are defined by the people who use them, not by dictionaries, and the sole criterion is effective communications. : Language is not science and people arrive at word definitions by. Dictionaries are not meant to tell people how to use words, but to. For example, let's look at the definition of "literally": Merriam-Webster — literally: Definition (2) flatly contradicts definition (1). The reason? When words are used in contradictory ways, a dictionary can only report this fact — so it does. Words are defined by the people who use them, not by dictionaries, and the sole criterion is effective communications.

Because words don't have fixed meanings (see "Word Definitions" box on this page), my definitions for theory, hypothesis and speculation are obviously open to debate, but I think these definitions are consistent with common usage, and provisional agreement on these terms is important to the topic. However, because the meaning of scientific theory is critical to science, its definition is less open to debate — a theory must be supported by evidence and must be falsifiable.

Some fields are accepted as scientific, some are not. The distinction between the two depends on the presence or absence of testable, falsifiable theories and a few other things:

A scientific field is defined by its theories. No theory, no science, no scientific field. If a field doesn't have a central, testable, falsifiable corpus of theory that all work in the field addresses, the field is not scientific. If a field's theories cannot be tested and falsified, the field is not scientific. If a field's theories don't have supporting evidence, or if the evidence falsifies the theories, the field is not scientific. If research in the field does not address the field's theories, that research cannot confer scientific status to the field.

The implications of item (5) above are far-reaching and suggest that many fields commonly thought to be scientific, fields with scientists, results and scholarly journals, are scientific in name only. For example, much of the scientific work in psychology cannot confer scientific status to psychology itself on the ground that it doesn't address psychology's central defining theories. The reason for that, in turn, is because psychology doesn't have a central, clearly defined theoretical structure open to test and falsification. This defect is shared by many of the "social sciences," fields described as sciences only to confer an unearned status.

To make this point, let's compare psychology with physics.

Physics Example

The Global Positioning System (GPS) is a reliable way to establish one's position on Earth's surface. The GPS system relies on satellites carrying very accurate atomic clocks. The satellites send radio signals to a user's GPS receiver, which calculates a position based on the arrival times of the satellite signals.

What transmitter power the satellites must have to be successfully received by GPS receivers on the surface.

How much thrust a particular rocket fuel can produce, and how much that fuel weighs.

Which orbit is optimal for a GPS satellite.

How large a rocket must be to successfully launch the satellite and place it in the desired orbit.

Which GPS clock timing to set, based on:

The satellite's orbital speed (to account for the time change resulting from Special Relativity) and



The satellite's position in Earth's gravitational field (to account for the time change resulting from General Relativity).

How to produce and maintain the desired stable orbit, which requires a deep understanding of orbital mechanics.

To launch a GPS satellite, one must have a deep understanding of physical theory:

This example shows the importance of theory to a practical result, the central role played by theory in physics, and the testability and falsifiability of the theories. Specifically, the designers of the GPS system knew in advance that the atomic clocks carried by the GPS satellites needed to be adjusted to accommodate the effects of both Special and General Relativity. Upon launching the satellites, the prediction was confirmed and the system met or exceeded its design goals — and coincidentally provided another experimental confirmation of physical theory.

Because physics is a science, because it is based on tested theories, physicists can reliably say what will happen in a given situation, and more important, they can say why — in other words, they can move beyond description to explanation.

Some have argued that describing is enough to create science, and there are fields entirely based on description, but to shape a theory one needs an explanation:

In explaining an observation, one takes the first step toward a theory.

A theory can be used to generalize a specific observation.

The theory can be used to predict similar results in other circumstances within the theory's domain.

The predictions can be tested and will either confirm or falsify the theory.

This is why description alone cannot lead to science — without an explanation, there is no basis for generalization, prediction, testing, and falsification.

Psychology Examples

My use of psychology as a counterpoint isn't to argue that it's a particularly bad example of sloppy science and pseudoscience, only that I've studied psychology extensively over the past five years as part of an effort to persuade psychologists that a more scientific approach might be in their best interests.

Falsifiability is essential to a theory's scientific standing. This doesn't mean all theories are false, it means a legitimate scientific theory must be testable in practical experiments, and potentially falsified by those tests: If a theory cannot be falsified, it is not scientific.

If a field is defined by an unfalsifiable theory, the field is not scientific.

If a position is based on an unfalsifiable theory, that position is not scientific. Falsifiability is the single most important property of science and scientific thinking. is essential to a theory's scientific standing. This doesn't mean all theories are false, it means a legitimate scientific theory must be testable in practical experiments, and potentially falsified by those tests:

Psychology is not defined by central, testable theories, and it is almost entirely reliant on description, not explanation. When a psychologist makes a claim — for example, "Cognitive-Behavioral Therapy (CBT) is more effective than its alternatives" — it is a description without any effort to explain. If someone were to try to explain why CBT is effective, that explanation might lead to a theory that could be tested in different, complementary circumstances, and the possibility of a practical test would create the falsifiability criterion on which science depends (see the "Falsifiability" box on this page).

As a result of psychology's theory vacuum, if someone contradicts the claim that CBT is more effective than alternatives (e.g. more effective than speaking to a sympathetic aunt), this counterevidence would make no difference to psychology or the practice of CBT. I know this to be true, because that refutation has been made repeatedly and supported by research[1],[2],[3], but these results have had no effect, on the ground that an experimental result cannot refute a nonexistent theoretical claim.

"A meta-analytic review of interventions based on MI found effect sizes across studies in the small to moderate range for alcohol and the moderate range for drug use when compared to a placebo or no-treatment control group ..."

Other serious obstacles to psychology's scientific standing are poor experimental controls and nearly nonexistent replication rates. Here is an example of poor experimental controls drawn from psychology's professional literature

Excuse me? What is a "no-treatment control"? Well, it's a group of people who are not given any treatment. Their outcome is being compared to a group of people who are given treatment. This experimental design is typical of modern psychological research — the "control group" are people who are simply told to go home, while the experimental group receive one-on-one sessions with a mental health professional. The outcome is expected to have scientific validity.

The Placebo Effect is a phenomenon in which an ineffective treatment (a sugar pill) produces a seemingly beneficial effect due to psychological factors, or an objectively beneficial treatment is undermined by psychological factors. The Placebo Effect is very powerful and represents a confounding obstacle to research involving human subjects. is a phenomenon in which an ineffective treatment (a sugar pill) produces a seemingly beneficial effect due to psychological factors,an objectively beneficial treatment is undermined by psychological factors. The Placebo Effect is very powerful and represents a confounding obstacle to research involving human subjects.

Because of the Placebo Effect (see the "Placebo Effect" box on this page), the described study is less than pointless. It is pointless because the control group's treatment is so distinct that they cannot realistically be thought of as a control, and it is less than pointless because the result has been published as though it has scientific meaning. This study joins a huge corpus of research of similar quality that represents the present state of psychological research.

For those unfamiliar with human studies, here is a list of experimental designs ranked in descending order by their probability of producing useful science:

Prospective studies, studies in which groups are randomly selected from a representative population and experimented on:

A "double-blind" controlled experiment is one in which neither the researchers nor the subjects know which group (experimental or control) they belong to, and ideally those who later evaluate the experimental data also do not know which group is which.



A "single-blind" controlled experiment is like the above but the experimenter knows which subjects are experimental and which are controls.



A controlled experiment is one in which an experiment is conducted on two groups — a group receiving the stimulus under study, and a control group not receiving that stimulus — but no effort is made to conceal the identities of the groups.



An uncontrolled experiment is an informal study in which a stimulus is applied to one group, and there is no control for comparison purposes.

Retrospective studies, studies in which the experimental groups are selected from within the population based on their past histories:

Studies in which two groups can be located in the population that are similar except for the trait under investigation.



Studies in which one population can be located and is compared to the general population.



Studies in which conclusions are drawn on the basis of popular accounts and common knowledge.

The problem with retrospective studies is that there is no meaningful way to draw reliable conclusions based on them. For example, let's say a study is meant to determine whether a group that takes vitamins is more intelligent than one that doesn't. We can't just sign people up for a prospective study and give half the subjects vitamins and the other half sugar pills — that would be unethical. So we must use a retrospective experimental design, one in which the subjects are drawn from the population, based on their pre-existing behaviors — some who take vitamins, some who don't — and try to decide how this affects intelligence.

Most readers will see the problem with this design — for groups drawn from the population at large, some of whom take vitamins, and some who don't, how are we to determine whether the experimental outcome is a cause or an effect? Did the subjects take vitamins because they are intelligent, or are they intelligent because they take vitamins?

Study: Pot Smoking Increases Risk of Psychosis[4]. A quote: "Of those who smoked pot for more than six years, 3 to 4 percent went on to develop a psychotic disorder before the age of 21. By comparison, lead study author John McGrath estimates that around 1 percent of people worldwide suffer from psychotic ailments."

This example shows the key problem with retrospective experimental designs — one cannot reliably separate causes and effects. And the number of "studies" that try to draw scientific conclusions from observed public behaviors is staggering, and the undeserved attention these studies get is no less than a scandal. Here's an example:

To avoid the pitfalls of a retrospective study, the researchers briefly considered asking teenagers to smoke pot, but quickly saw the problems with this approach. This means the classic problem endemic to retrospective studies is present in this study — a confusion of cause and effect (people inclined to use drugs might be more prone to mental illness). Well below the scare headline are comments like this: "... every study on the issue thus far has been imperfect: Despite controlling for variables like family history or childhood trauma, researchers were hard pressed to conclude that marijuana use caused psychosis, and not the other way around." Or both these factors might be correlated with a third unexamined possibility.

This quote is also noteworthy: "Armentano also points to studies that have found no connection between pot and psychosis, like a systematic review out of the United Kingdom just last year. 'I don't see any Reuters headlines on those,' he said, noting that it's much easier to get research dollars for studies into the adverse effects of marijuana."

So, if the scientists behind this study openly acknowledge that a cause-effect relationship cannot be established, how does the article merit the headline, "Pot Smoking Increases Risk of Psychosis"? The answer is that this is typical of modern-day psychological research. Psychology cannot become truly scientific until there is a defining, falsifiable theory to which all such work refers, and until psychologists begin explaining instead of describing. Until this happens, no amount of psychological research can grant psychology the status of a science.

I chose a drug study for this example because such studies are especially difficult for practical reasons, and because the topic is controversial, bias becomes a risk at every level — from acquiring grant money, to choosing subjects and protocols, to the interpretation of results.

Consider the implications of this example. On one hand we have an experimental design that cannot produce a reliable result and a study that ends up producing an ambiguous outcome. On the other hand, we have public information machinery that can be relied on to mischaracterize a study's outcome and as a side effect make science look like a tool for confirmation of popular sentiment.

Describing versus Explaining

The distinction between describing and explaining is critical to science and the shaping of theory:

For a field to become scientific, it must have one or more theories.

The theories must be testable in practical experiments and potentially falsifiable.

To support a field's scientific status, research within the field must address that field's theories.

Theories are shaped by creating an explanation for observations, then generalizing the explanation, then testing the generalization in new experiments.

No amount of description can substitute for an explanation.

Here are some examples that show the advantage of explanation over description:

One day at the seashore, the water level suddenly drops and recedes from shore in an unprecedented way, uncovering many fish suddenly stranded, flopping about on the sand, waiting to be picked up.

Description : The ocean has receded, and stranded fish can be picked up off the sand.

: The ocean has receded, and stranded fish can be picked up off the sand.

Explanation : A sudden drop in water level at a beach is a classic warning of an impending tsunami , a tidal wave, and to avoid drowning in the oncoming wave, people must move to high ground immediately.

: A sudden drop in water level at a beach is a classic warning of an impending , a tidal wave, and to avoid drowning in the oncoming wave, people must move to high ground immediately. Joe, a teenage driver, notices that his car needs 80 feet to stop when traveling at 40 miles per hour. Joe wants to know how much stopping distance he will need at 80 miles per hour. (For simplicity, we neglect reaction time in this example.)

Description : Common sense says that, if 80 feet is needed at 40 miles per hour, then 160 feet will be needed at 80 miles per hour.

: Common sense says that, if 80 feet is needed at 40 miles per hour, then 160 feet will be needed at 80 miles per hour.

Explanation : A moving object has kinetic energy [5] , and it is this kinetic energy that is dissipated by tire friction on the roadway. Kinetic energy is equal to an object's mass times the square of its speed , so if you double a car's speed, its stopping distance becomes four times greater — so the stopping distance required for 80 miles per hour is not 160 but 320 feet — twice the expected distance based on "common sense".

: A moving object has kinetic energy , and it is this kinetic energy that is dissipated by tire friction on the roadway. Kinetic energy is equal to an object's mass times , so if you double a car's speed, its stopping distance becomes — so the stopping distance required for 80 miles per hour is not 160 but 320 feet — twice the expected distance based on "common sense". A group of fishermen is rescued from the North Sea, where they have been immersed in cold water for 90 minutes. They are severely hypothermic. What is the best aid strategy?

Description : The rescued men are still very cold, and that cold may kill them, so warm them up — give them warm blankets and hot drinks.

: The rescued men are still very cold, and that cold may kill them, so warm them up — give them warm blankets and hot drinks.

Explanation : Hypothermia is as complex as it is dangerous. The biggest danger is to allow cold blood from the extremities to rush back into the victim's torso too quickly, and the easiest way to provoke this response is to warm the victim too fast.

: Hypothermia is as complex as it is dangerous. The biggest danger is to allow cold blood from the extremities to rush back into the victim's torso too quickly, and the easiest way to provoke this response is to warm the victim too fast.

(This is a true story[6], and the result was tragic — In 1980, 16 Danish fishermen were rescued from the North Sea and given hot drinks. Within an hour, all of them were dead.)

To summarize, fields that describe natural phenomena are not sciences unless they endeavor to explain what they describe. The explanations can lead to a theory, and it is testable theories that define a scientific field. This means one may find an organized, distinct field, with scientists doing legitimate scientific research and publishing in scientific journals, but unless the research addresses that field's theories, the research cannot contribute to the field's scientific status.

No testable theory → no science → no scientific field — regardless of the number of clipboards and white lab coats.