In 1915, Albert Einstein, with a little help from his friends, developed a theory of gravity that overturned what we’d thought were the very foundations of physical reality. The idea that the space that we inhabit was not perfectly described by Euclidean geometry had been inconceivable—so much so that the philosopher Immanuel Kant, a radical thinker in so many ways, proclaimed that it was not possible for any theory of physics to dispense with it.

The physicist Werner Heisenberg later pointed out the implications of Kant’s mistake. The great philosopher had posited that our intuitions about the ancient geometry of Euclid meant that it was the necessary foundation for physical reality. The fact that this turned out to be false called the integrity of Kant’s entire philosophical edifice into question.

Despite their radical break with past ideas of space and time, Einstein’s theories would soon end up lumped in with Newton’s as part of “classical physics.” We needed to do so because there was a revolution in scientific thought so profound that it created a bright line in the history of science: the development of quantum physics.

What could qualify as a scientific revolution more profound than the Theory of General Relativity? What could have created a seismic shift more violent than the idea that space and time themselves were curved and bent by matter?

To understand that, we must first try to understand: the essential bizarreness of quantum mechanics. Once the quantum world leaves us feeling suitably uncomfortable, then we can understand why, ever since it arrived on the scene, physicists have been trying to construct alternatives to quantum mechanics—alternatives that reproduce the same fantastic agreement with experiments while retaining part of the classical core that agrees with our deepest intuitions about how nature should behave.

Everything you know is wrong

Our deepest intuitions about the nature of reality are built up as we observe and interact with the world around us, beginning as infants. Before we can articulate it, we begin to understand cause and effect. Everything that happens was caused to happen by something else. The world is predictable.

Later, we become sophisticated. We accept that there are limits to our knowledge of causes and recognize uncertainties about their effects. We may even study probability and statistics and learn to express the limits of our knowledge in mathematical form. But we assume that these limits are ours and that, unseen to us, nature continues to apply exacting rules of cause and effect behind the scenes. When we toss a coin, it’s our lack of information about its motion and that of the air that resigns us to saying no more than that there is a probability of one-half that it will land heads-up. We assume that if we did know the details and had a big enough computer to perform the calculations, then we would not have to resort to probabilities.

This “realistic” view cannot, (and did not), survive the onslaught of data from experiments on photons and other subatomic particles. It’s not that physicists, in a fit of stubborn perversity, decided to construct a theory that contradicted our most cherished intuitions about reality. Instead, the results of experiments stubbornly refused to yield to any sort of classical interpretation. The invention of the quantum formalism was an act of desperation—one that worked. If we limit ourselves to asking questions permitted by quantum theory, we’ll be rewarded with correct answers. But if we insist on trying to grasp the meaning of what the theory tells us using concepts from the classical world, we’ll become mired in confusion.

As a physics student, I saw a classroom demonstration that gave a brief glimpse of the unseen strangeness that is all around us. You can do this at home with nothing more than a flashlight or a laser pointer and three cheap polarizing filters (you can also use the lenses from broken polarized sunglasses). Place two filters in a row with a gap between them. Pass the light through the pair, and rotate one until no light gets through; their polarization axes are now perpendicular. Now insert the third filter between the first two. You will observe light passing through the assemblage; somehow adding an additional filter allows light to pass through.

The demonstration was part of an introduction to a course in quantum mechanics. Within a few weeks, we were immersed in a formalism from which this seemingly paradoxical behavior emerged as a trivial consequence.

There are those who claim that there is no paradox here and that the behavior can be explained classically. They are, in a sense, entirely correct. But the results of a tabletop demo, startling to students already familiar with classical physics, come out cleanly from the quantum formalism. And that probably tells us something.

Scientists in the first few decades of the last century were faced with experimental results far more startling and inexplicable than these. An often-discussed example is the two-slit experiment, shown here. Whether you perform the experiment with electrons or photons, the results are the same: you get an interference pattern, just as if two waves emerged from the two slits and interfered with each other. This shows that light is a wave and that even particles with mass, such as electrons, seem to behave as waves under these circumstances.

But the experiment lends itself to two curious developments. First, if you slow down the rate of emission of particles (photons, electrons, even entire molecules) so that only one can pass through the slits at a time, the result does not change. This must mean, somehow, that the particle splits into two, passes through both slits, and interferes with itself! Second, if you make any alteration to the apparatus to record which slit the particle passes through, the interference pattern disappears and is replaced by the pattern you would expect if the particles were simply particles, with no wavelike characteristics: two symmetrical distributions centered on each slit.

It was difficult to come up with a story that both explained results and that everyone could agree on. It seemed as if the photons or electrons sometimes decided to behave as waves and sometimes as particles, depending on what the experimenter decided to look at.

In more recent times, things have only gotten stranger. Technology has advanced to the point where we can decide what kind of measurement to take after the particle has begun its journey. The results of these “delayed choice” experiments are the same. If we look to see which direction the particle takes, the interference is destroyed. If we avert our eyes, so to speak, the familiar interference pattern returns. And yet, the particle must have “decided” to act like a particle or a wave before passing through the slits and before the final configuration of the experiment.

The results of the delayed choice experiments have led more than one physicist to speculate that the information about whether to be a particle or a wave is transmitted back in time, from the time of the choice to some time before the particle traverses the device. That this proposal has been seriously considered should give you some idea of how difficult it is to explain the results of experiments on the micro-world using a set of concepts (like causality) taken from our realistic worldview. The back-in-time explanation has been stretched to the breaking point recently by using slow, cold helium atoms in a similar setup. The atoms are pulled through the apparatus by the action of gravity alone, so significant time elapses between their transit and the choice of how to observe them. Although physicists sometimes describe certain very fast subatomic processes as involving a limited form of backward time travel, the long timescales in the helium experiments make this explanation impossible to support.

Where does this leave us? The results of these and many more experiments simply cannot be described using traditional, reality-based concepts: that objects exists with a certain set of properties; that if you choose not to measure a particular property, it still has a particular value. Physicists were well practiced in dealing with uncertainty long before the quantum revolution, but this is uncertainty of a very different kind. It was uncertainty of knowledge, that assumed an unknown, but real, level of deterministic reality below what could be directly apprehended.

If we throw out these ideas fundamental to our understanding of the world, what can we replace them with? It’s not just that they’re an intuitive part of everyday experience. They serve as the foundation for other fields of science.