SANTA BARBARA—"Maybe we're just too dumb," Nobel laureate physicist David Gross mused in a lecture at Caltech two weeks ago. When someone of his level wonders whether the unification of physics will always be beyond mortal minds, it gets you worried. (He went on to explain why he doesn't think we are too dumb, though.) Since his lecture, I’ve been learning about a theory that seems, at first, to confirm this worry. It is so ridiculously hard that it could be the subject of an Onion parody. But at the same time, I’ve been watching how physicists are trying to power through their intimidation, because the theory promises a new way of understanding what space and time really are, at a deep level.

The theory was put forward in the late 1980s by Russian physicists Mikhail Vasiliev and the late Efin Fradkin of the Lebedev Institute in Moscow, but is so mathematically complex and conceptually opaque that whenever someone brought it up, most theorists started talking about the weather, soccer, reality TV—anything but that theory. It became a subject of polite conversation only in the past couple of years, as math whizzes who take a peculiar pleasure in impossible problems dove in and showed that the theory is not impossible to grasp, merely almost impossible.

Inspired by their bravery, I'm going to take a crack at explaining this strange beast, synthesizing lectures I've attended by Steve Shenker of Stanford University, Andy Strominger of Harvard, and Juan Maldacena of the Institute for Advanced Study, as well as informal chats with Joe Polchinski of the Kavli Institute for Theoretical Physics and Joan Simón of the University of Edinburgh. I'm sure they'll set me straight if I get something wrong, and I'll edit this blog post to reflect comments I receive.

Vasiliev theory (for sake of a pithy name, physicists drop Fradkin’s name) takes to extremes the basic idea of modern physics: that the world around us consists of fields—the electrical and magnetic fields and a handful of others that represent the known forces of nature and types of matter. Vasiliev theory posits an infinite number of fields. They come in progressively more complicated varieties described by the quantum-mechanical property of spin.

Spin is perhaps best thought of as the degree of rotational symmetry. The electromagnetic field along with its associated particle, the photon, has spin-1. If you rotate it 360 degrees, it looks the same as before. The gravitational field along with its associated particle, the graviton, has spin-2: you need to rotate it only 180 degrees. The known particles of matter, such as the electron, have spin-1/2: you need to rotate them 720 degrees before they return to their original appearance—a counterintuititive feature that turns out to explain why these particles resist bunching, giving matter its integrity. The Higgs field has spin-0 and looks the same no matter how you rotate it.

In Vasiliev theory, there are also spin-5/2, spin-3, spin-7/2, spin-4, all the way up. Physicists used to assume that was impossible. These higher-spin fields, being more symmetrical, would imply new laws of nature analogous to the conservation of energy, and no two objects could ever interact without breaking one of those laws. The workings of nature would seize up like an overregulated economy. At first glance, string theory, the leading candidate for a fully unified theory of nature, runs afoul of this principle. Like a plucked guitar string, an elementary quantum string has an infinity of higher harmonics, which correspond to higher-spin fields. But those harmonics come with an energy cost, which keeps them inert.

Vasiliev and Frakin showed that the above reasoning applies only when gravity is insignificant and spacetime is not curved. In curved spacetimes, higher-spin fields can exist after all. Maybe overregulation isn't such a bogeyman after all.

In fact, it may be a positive good. Higher-spin fields promise to flesh out the holographic principle, which is a way to explain the origin of space and gravity. Suppose you have a hypothetical three-dimensional spacetime (two space dimensions, one time dimension) filled with particles that interact solely by a souped-up version of the strong nuclear force; there is no gravity. In such a setting, objects can behave in a very structured way. Objects of a given size can interact only with objects of comparable size, just as objects can interact only if they are spatially adjacent. Size plays exactly the same role as spatial position; you can think of size as a new dimension of space, materializing from particle interactions like a figure in a pop-up book. The original three-dimensional spacetime becomes the boundary of a four-dimensional spacetime, with the new dimension representing the distance from this boundary. Not only does a spatial dimension emerge, but so does the force of gravity. In the jargon, the strong nuclear force in 3-D spacetime (the boundary) is "dual" to gravity in 4-D spacetime (the bulk).

As formulated by Maldacena in the late 1990s, the holographic principle describes a bulk where dark energy has a negative density, warping spacetime into a so-called anti-de Sitter geometry. But this is just a theorist's playground. In the real universe, dark energy has a positive density, for a de Sitter geometry or some approximation thereof. Extending the holographic principle to such a geometry is fraught. The boundary of 4-D de Sitter spacetime is a 3-D space lying in the infinite future. The emergent dimension in this case would not be of space but of time, which is hard even for theoretical physicists to wrap their minds around. But if they succeed in formulating a version of the holographic principle for a de Sitter geometry, it would not only apply to the real universe, but would also explain what time really is. A lack of understanding of time is at the root of almost every deep problem in fundamental physics today.

That is where Vasiliev theory comes in. It works in either an anti-de Sitter or a de Sitter geometry. In the former case, the corresponding 3-D boundary is governed by a simplified version of the strong nuclear force rather than the souped-up one. By biting the bullet and accepting the borderline-incomprehensible Vasiliev theory, physicists actually end up easing their task. In the de Sitter case, the corresponding 3-D boundary is governed by a type of field theory in which time does not operate; it is static. The structure of this theory gives rise to the dimension of time. What is more, time arises in an inherently asymmetric way, which might account for the arrow of time—its unidirectionality.

It gets even better. Normally the holographic principle can account for the emergence of one dimension, leaving the others unexplained. But Vasiliev theory might give you the whole kit and kaboodle. The higher-spin fields possess an even higher degree of symmetry than the gravitational field does, which is a lot. Higher symmetry means less structure. The theory of gravity, Einstein's general theory of relativity, says that spacetime is like Silly Putty. Vasiliev theory says it is Sillier Putty, possessing too little structure to fulfill even its most basic functions, such as defining consistent cause-effect relations or keeping distant objects isolated from one another.

To put it differently, Vasiliev theory is even more nonlinear than general relativity. Matter and spacetime geometry are so thoroughly entwined that it becomes impossible to tease them apart, and our usual picture of matter as residing in spacetime becomes completely untenable. In the primordial universe, where Vasiliev theory reigned, the universe was an amorphous blob. As the higher-spin symmetries broke—for instance, as the higher harmonics of quantum strings become too costly to set into motion—spacetime emerged in its entirety.

Perhaps it is not so surprising that Vasiliev theory is so complicated. Any explanation of the nature of space and time is bound to be intimidating. If physicists ever do figure it out, I predict that they'll forget how hard it used to be and start giving it to their students for homework.