When Brian Schmidt got his PhD in astrophysics in 1993, he was one of less than a handful of people that year that graduated with a thesis on supernovae. Five years later, still working on exploding stars, he would be part of one of two teams that independently discovered that the universe was not only expanding, but that its expansion was accelerating.

That the expansion of the universe is accelerating means it is being pushed apart by some kind of energy embedded in the fabric of space itself. This energy makes up over 70% of the universe. We call it dark energy, mainly because we are in the dark about what it actually is.

The 2011 Nobel Prize in Physics was awarded to Brian Schmidt, along with Adam Reiss, who worked with Schmidt as part of the High-z Supernova Search Team, and Saul Perlmutter, who headed the rival Supernova Cosmology Project, for their discovery of the accelerating expansion of the universe. The announcement of the prize called dark energy “perhaps the greatest enigma in physics today”.

When Schmidt attended the 62nd Lindau Nobel Laureates Meeting in Germany earlier this month, I met him and found out more about the discovery.

Big questions

“I liked the fact that supernovae changed,” Schmidt says of his PhD topic. “That appealed to me.”

For his PhD, Schmidt developed a way to measure the distances to type II supernovae, explosions of massive stars that have come to the end of the main part of their life. Type II supernovae differ from other types in that they have hydrogen in the spectrum of light we detect coming from them. He used these distance measurements to calculate a number called the Hubble constant that you can use to work out the age of the universe. The idea was that you can use this to look back and work out how fast the expansion of the universe is slowing down – because, at the time, that’s what people thought was happening. “It tells you the ultimate fate of the universe. Gosh, that was a big question,” says Schmidt. “I loved it.”

By 1994, after Schmidt's PhD, technology had advanced so much that supernova distances were now measurable accurately. “[The Supernova Cosmology Project] had been looking for supernovae for six years and suddenly were able to find them,” says Schmidt. The Supernova Cosmology Project had started in 1988, with Saul Perlmutter at its head. The project was looking for type Ia supernovae, which result from the explosion of a white dwarf – a star that is itself a remnant of a star, no bigger than about one a half times the mass of the sun, that has stopped fusing hydrogen and helium. The light from type Ia supernovae always follows the same pattern, meaning astronomers can work out how far away one particular stellar explosion is by its brightness (for example, see this).

“We talked about working with them,” says Schmidt. But both groups had strong views over how the work should be carried out. “We had disagreements and it was very clear that they did not want us involved in their experiments,” he added.

But Schmidt still felt that what he wanted to do was the right thing to do at the time. So he dropped everything else and did it. Along with Nicholas Suntzeff, he formed the High-z Supernova Search Team in 1994 to compete with the Supernova Cosmology Project in tracing the expansion of the universe using type Ia supernovae. Adam Reiss took charge of the High-Z team's data. At the time Schmidt was supposed to be working on another project, but abandoned it. “There was no sense in working on anything else because this was what I wanted to do,” he says.

“I remember saying that if those guys can find them we can,” says Schmidt. He describes himself at that time as young, arrogant and naïve. “It’s always good to be naïve,” he says. It was a bold move. Luckily, the risk paid off.

First light

Soon the High-z Supernova Search Team got their first data set, from supernova 1995K. By September 1997 they were still gathering data, but nothing looked amiss just yet. “At that point it was consistent with something reasonable,” Schmidt says.

But by the end of 1997 it became clear that the High-z Supernova Search Team’s results were anything but reasonable. The supernovae seemed to be telling them that the expansion of the universe, far from slowing down, was in fact speeding up. “Adam [Reiss] put all that data together and he sent me a figure,” says Schmidt. “It said ‘What do you think of this?’ as the subject line of the email and that was it.”

Schmidt and Reiss spoke on the phone to discuss the paradigm-shifting figure Reiss had emailed. They decided to go through every step of the data analysis again before talking to anyone else about it. “My initial reaction was that we must have made some kind of mistake,” says Schmidt.

They started that painstaking process at the end of November. “I was learning this stuff on the fly at this point because it was very clear that we had to do something new,“ says Schmidt. By the beginning of January they agreed on “every single bit” of the analysis. “I remember the moment when I thought: We’re going to have to tell everyone in the team about this. And then we’re going to have to tell the world about it at some point,” he says.

Some members of the research team were understandably uneasy with the findings. Schmidt recalls telling them: “I find it disturbing too, but I’ve been working on it for the last six weeks and can’t make it go away.” The team was able to come up with some tests that Schmidt and Reiss hadn’t done. So they did those over the next couple of months.

But the accelerating universe was still there.

Closing in

The whole time that Schmidt’s group was working on checking and rechecking their analyses they knew that Saul Perlmutter’s group were working on the same problem as them, trying to measure the expansion of the universe with supernovae. But they had thought that their rivals were getting the opposite answer, one that said the expansion of the universe was slowing down, as everyone expected.

They weren’t. On 8th January 1998, Saul Perlmutter, on behalf of the Supernova Cosmology Project, presented data at the American Astronomical Society meeting in Washington, D.C., that hinted at an accelerating universe. The next day, Charles Petit at the San Francisco Chronicle reported the finding on the front page of the newspaper. This was the first that anyone at Schmidt’s group had heard about the Supernova Cosmology Project's data.

“That certainly made us focus our attention,” says Schmidt, who didn’t make it to the AAS meeting himself. “We went charging ahead. We showed the world our results on 23rd February. [Perlmutter’s group] were completely surprised by it,” says Schmidt.

When they announced their evidence for the accelerating universe, Schmidt was worried they would be “crucified”. But the fact that there were two teams that independently came to the same result at the same time was a big deal, he says.

Fixing problems

The other thing that helped the result gain acceptance was that it solved an awful lot of problems. “It had been known for the previous seven or eight years that a cosmological constant, if it existed, would explain several of the anomalies in the cosmological model at the time,” says Schmidt. A cosmological constant is the simplest possible description of dark energy, the force causing the acceleration of the universe. It was, bizarrely enough, first suggested by Albert Einstein as a way to make his equations describe a static universe. He quickly dropped his version of the cosmological constant when astronomers in the 1920s discovered that the universe was not static after all, but expanding. But since the discovery of the accelerating universe and dark energy, a cosmological constant term has found favour once again as a way to make the equations describing the universe fit with what astronomers see. Adding a cosmological constant essentially has an anti-gravity effect on the universe, allowing it to push itself apart when it should be slowing down.

Many aspects of cosmology suddenly made sense when a cosmological constant was added to the mix. “The whole inflation scenario was easily explained if there was a cosmological constant,” Schmidt says. “It made large scale structure work. It made the universe flat, it made the universe the right age. It meant that the Hubble constant could be up where we measured it. It meant that [the universe’s matter density] could be where we measured it. All the things worked.”

The High-Z Supernova Search Team had been careful not to announce a discovery, but rather observational evidence. Both they and the Supernova Cosmology Project had three-sigma confidence, meaning that the probability their results were down to chance was 0.27%. Together they had four and a bit sigma – giving an even smaller probability that it was down to chance. But still big enough for some people to worry about. So they worked in the years afterwards to build up the proof.

The reality of the finding really sunk in for Schmidt in 2000, he says. Two separate experiments, MAXIMA and BOOMERANG, that had both measured photons left over from the early universe, known as the cosmic microwave background, had shown that the universe was flat. Their findings fitted in with a universe that had undergone an early period of fast inflation, contained that elusive stuff dark matter – and, most importantly, required a cosmological constant. “I just couldn’t see any way to get rid of it at that point,” Schmidt says.

Missing universe

“The only problem with the standard model of cosmology is that it requires us to invent 95% of the universe”

Fast forward to last month and Schmidt was presenting his lecture at the 2012 Lindau Nobel Laureates meeting. If anyone hadn’t quite woken up for the first lecture on Monday morning, that sentence was sure to do the job. Schmidt was referring to the fact that only about 5% of the energy density of the universe is made up of ordinary matter like atoms. Some 23% is dark matter. The remaining 70%+ is the dark energy driving the universe’s accelerating expansion. “I still don’t like it,” Schmidt says. “But it works, and ultimately, it’s not so ugly that we shouldn’t live with it.”

We may have to live with dark energy, but finding out what exactly it is will prove an incredible challenge. The cosmological constant is just one idea, and even that is more of a starting point than a final answer. It needs digging into in a lot more detail to get a better description of the state and future of the universe. But that will not be easy. “The reality is it’s going to be very difficult to get our hands on that dark energy sector. Because we’re not going to create it in the lab, that’s the problem,” says Schmidt.

He does have some ideas, however. “If I could do my big experiment to measure the equation of state parameters [the numbers that cosmologists plug into equations that describe the universe] I’d use baryon acoustic oscillations.”

Baryon acoustic oscillations are fluctuations in the density of matter caused by acoustic waves in the early universe. They left an imprint in the clustering of galaxies and matter today. We know how big they should have been in the early universe, so by looking at the separation of galaxies today astronomers can work out how far away those galaxies are – and map the expansion of the universe.

“But I’m going to let someone else do that,” says Schmidt.

From 1st to 6th July I was at the Lindau Nobel Laureates Meeting in Germany as part of the Lindau blog team. You can read my posts from there here or on the Lindau blog.

*

Updated 5th August 2012, to reflect Nicholas Suntzeff's role in the formation of the High-Z Supernovae Search Team (see his comment).