“[The] concept of the cosmos that [Albert] Einstein derived form this general theory of relativity was elegant and magical. But there seemed to be one hitch, a flaw that needed to be fixed or fudged. His theory indicated that the universe would have to be either expanding or contracting, not staying static. According to his field equations, a static universe was impossible because the gravitational forces would pull all the matter together.” – Walter Isaacson, Einstein: His Life and Universe, 2007, p. 254.

On February 8, 1917, Einstein published a paper titled ‘Kosmologische Betrachtungen zur allgemeinen Relativitätstheorie‘ (‘Cosmological Considerations in the General Theory of Relativity’). In it, he described a number called the cosmological constant. The constant had a value such that, when used in his newly created equations describing the behaviour of the gravitational force, a non-changing universe was spit out – agreeing with knowledge at the time, as well as his belief, that the universe was static. Without the constant in the picture, on the other hand, Einstein’s general theory of relativity suggested that the gravitational pull of masses contained in the universe would pull all the matter together, keeping the universe dynamic.

It would be more than a decade before evidence would begin to emerge that the universe was expanding. And it would be scores of years before astronomers would find that the expansion was also accelerating. Then again, it would be many years before Einstein realised his actual mistake.

While he popularly considered his addition of the constant to be an affront to his own work, it may not have been as bad as he thought. The cosmological principle states, rather assumes, that the universe at the largest scales has the same properties everywhere. This is a spatial definition. An extension called the ‘perfect’ cosmological principle states that the universe at the largest scales has the same properties everywhere and at every time, implying that it has always remained the way it is today and it will continue to be this way forever. This is also called the steady-state theory, an alternative to the Big Bang theory that has been widely discredited – and which Einstein himself pursued for a while in 1931.

Anyway, in defence of Einstein, theoretical astrophysicist Peter Coles writes on his blog, “General relativity, when combined with the cosmological principle, but without the cosmological constant, requires the universe to be dynamical rather than static. If anything, therefore, you could argue that Einstein’s biggest blunder was to have failed to predict the expansion of the Universe!” Indeed, if Einstein had not decided to fudge his own monumental equations, he may have been onto something.

As Coles goes on to explain, the field equations, like all equations, have two sides: on the left are the effects of gravity due to the bending of spacetime; on the right are the effects of massive bodies and their motion through spacetime. Einstein added the cosmological constant to the left side, indicating that he thought it was gravity’s side of things that needed to be modified to imply a static universe. Modern cosmologists, however, use the term on the right side, to indicate that it is matter and its behaviour through spacetime that need a modification. As a result, the cosmological constant is often related to the energy of spacetime itself, also known as dark energy or vacuum energy.

A flat universe

In 1998, two astronomy teams found that the universe’s expansion was accelerating. They were able to independently come to the same conclusion by observing stars around the universe that were dying as Type Ia supernovae. When a mid- to low-mass star runs out of hydrogen to fuse, it will attempt to fuse carbon. However, if it can’t become sufficiently hot to commission the fusion process, the star will accumulate carbon and oxygen in its core. Over time, the outer layers will dissipate to leave behind a carbon-oxygen lump called a white dwarf. If it has a binary companion – an object, such as another star, orbiting it – then the white dwarf could start siphoning material from the object’s surface, but only upto a point.

In 1930, a 19-year-old Indian astrophysicist named Subrahmanyan Chandrasekhar had found that white dwarf stars should blow up as soon as they became more than 1.44-times as heavy as our Sun. This explosion is called a Type Ia supernova. Because it is moderated by the Chandrasekhar limit, its characteristics – such as its brightness – are always the same no matter whatever else happened before the limit was breached. The two astronomy teams were able to use this information to determine how far various white dwarfs were in the universe and, using the Doppler effect, how quickly they seemed to be moving away.

They found that the cosmological constant was positive. More specifically, assuming that the energy density of our universe was close to its critical density, they calculated the value of a number called omega-lambda to be about 0.7. The universe’s critical density determines how information behaves within the universe. For example, if the universe’s density is higher than its critical density, then astrophysicists predict it will be shaped like a sphere, and that particles that originate on one point of it will eventually circle back to it if they persisted for long enough – and the information will reset and ‘restart’. Our universe is assumed to be flat, “like a sheet of paper”, because its density is lower than the critical density. And in a flat universe, omega-lambda is the fraction of dark energy. This is how we say, “The universe is 70% dark energy”, etc. – because omega-lambda hovers around 0.7.

As is evident, there are multiple implications of the cosmological constant that make Einstein’s anxious mistake so ironically important in the study of the universe. Yet another example of this is the so-called Lambda-CDM model, which is a set of theories and principles that together strive to describe the universe’s properties. Its name itself is made up of two things that Einstein helped us discover: the ‘lambda’ refers to the cosmological constant and CDM stands for ‘cold dark matter’, which is thought to interact with normal matter only through the gravitational force.

However, irrespective of its internal consistency, the LCDM model has off late been challenged for its shifting attempts to explain dark matter. Most recently, a Dutch physicist named Erik Verlinde rooted for a modified theory of gravity to altogether explain away the need for this enigmatic substance, although his claims have been disputed.

On second thought, LCDM is a pseudo-pitstop to take a break at and mull the cosmological constant; a more meaningful place would be the philosophy of science itself. It is evident that the constant and its relationship to the number called omega-lambda are consequential to information theory, the study of how information flows in this universe – from one end of time to another, through materials and fields, into and out of blackholes. It also affects quantum mechanics and, in one way, string theory.

While observational studies have found that the constant is a really small number between 0 and 1, quantum mechanics predicts that the number should be 10120 higher! The size of the hierarchy problem in particle physics pales in comparison, but more importantly, the discrepancy hints at some violation of the idea of naturalness that some physicists are fond of. Brian Greene spun this for his cause in The Elegant Universe: “This presents a wonderful challenge and opportunity for string theorists: can calculations in string theory improve on this mismatch[?] … Should string theorists be able to rise to this challenge – as yet they have not – it would provide a compelling piece of evidence in support of the theory.”

The constant’s origins themselves were rooted in philosophical considerations – they had to be because its inclusion in the field equations had been arbitrary. In a paper published in 1919, he stated that the term was “gravely detrimental to the formal beauty of the theory” – significant in opposition to his belief that simple was beautiful, as elucidated in a lecture in 1933: “nature is the realisation of the simplest that is mathematically conceivable”. Finally, neither Einstein nor the physicists of today have been able to explain why the constant is necessary at all and why the theory can’t account for the universe’s expansion without it.

The ideas of Ernst Mach

But Einstein only sowed as much dissonance because he was an adherent of the philosophy of the Austrian phenomenalist Ernst Mach and his so-called Mach principle. Mach had a problem with Newton’s second law of motion, that a body’s acceleration is proportional to the force acting on it, because the Englishman hadn’t specified a frame of reference against which these observations would be true. Although Newton had invoked centrifugal and Coriolis forces when proving his second law in the context of one body rotating around another – like a satellite around Earth – his defence also relied on the existence of ‘absolute space’.

In other words, it mattered to Newton if Earth was going around the satellite or the satellite was going around Earth when working out the answer. However, space is invisible and Mach disliked its use because his philosophy of science was that only that which could be perceived by human senses could be true. For him, moreover, it didn’t matter if the planet was orbiting the satellite or vice versa. It only mattered that both situations shared a common geometry, which in turn morphed into the principle in 1872 that Einstein was inspired by later.

But as fate would have it, Einstein’s field equations, born 100 years ago last year, ended up postulating a universe that would either have to expand or contract for all eternity. And the introduction of the cosmological constant was a last-ditch attempt to stall this overwhelming outcome, to prevent invisible energies from hijacking what was already a four-dimensional spacetime continuum – all antithetical to Mach’s view of the universe. In a review paper, Cormac O’Raifeartaigh, Michael O’Keeffe, Werner Nahm and Simon Mitton, physicists from Ireland and the UK, wrote,

Where he intended to introduce a term to the field equations representing an attenuation of the gravitational interaction at large distances, he in fact introduced a term representing a very different effect. Indeed, the later interpretation of the cosmological term as representing a tendency for empty space to expand would have been deeply problematic for Einstein in 1917, given his understanding of Mach’s Principle at the time. Thus, while there is no question that relativity allowed the introduction of the cosmic constant, it appears that Einstein’s interpretation of the term may have been to some extent founded on a misconception.

Just as well, Einstein’s desperation did not work but in fact backfired: he would shed his faith in the Mach principle in the early 1930s and the cosmological constant would become a firm part of modern cosmology.

Note: This article was edited on February 9, 2017, to reflect doubts about attributing the phrase “biggest blunder” to Albert Einstein.