In 2011, the ecologist Ryan Chisholm was looking at tree census data from 12 different forests around the world. More than 4,000 species of trees grew in these places, their numbers rising and falling over the years. The pictures the numbers painted were of ecosystems where a species’ fortunes could change nearly overnight, on an ecological timescale. For instance, a small, glossy-leaved tree called Inga marginata had 400 individuals in a Panamanian forest plot in 2005; by 2010 it had nearly doubled its numbers.

In all 12 forests, however, one detail was particularly notable. The speed and magnitude of the changes didn’t look anything like what would have been predicted by one of the leading theories in theoretical ecology. Models based on that idea, called neutral theory, have shown that the distribution of species over the landscape can be explained using surprisingly simple inputs. But here the theory was breaking down. “You look at how big these fluctuations are,” Chisholm said. “And they’re just enormous. They’re so much bigger than what neutral theory would predict. … Orders of magnitude bigger.”

When Chisholm gave a talk at the Smithsonian Tropical Research Institute in Panama, where he was a postdoc, he learned that other people had noticed the same thing. Whatever its successes, neutral theory did not model change well at all — even its estimates of how long it would take a species to go extinct could be tens to hundreds of times longer than the reality. A flurry of papers from various groups since then, including one by Chisholm and collaborators appearing yesterday in Ecology, look to answer the question: Can neutral theory be adapted so that it shows changes over time? And is it possible to link a beautifully simple model more closely with the complex messiness of biology without damaging the model?

In a neutral model, each individual in an ecosystem, regardless of the species, begins with the same fitness. As the years pass, metaphorical dice are rolled for each individual to decide whether it dies or reproduces. New individuals arrive from outside the community’s borders, and every now and then a new species arises. Over time, the results of these processes accrete in this pocket universe. The model includes nothing about the differences between species, or about niches where one species might thrive while another fails, or about whether a hurricane or drought has struck — it just posits a certain preprogrammed randomness. After a while, ecologists studying this pocket universe might push the pause button and see how the area has evolved. Depending on how they’ve set up the model, the ecologists could look at how many individuals there are of each species, for example, or how the species have arrayed themselves across the landscape.

For certain places, including tropical rainforests and coral reefs, the picture generated in this way looks strikingly similar to the observations of field biologists counting trees in remote jungles or coral species on the ocean floor. “Neutral theory was a hit because it proposed that a lot of nature’s complexity can be approximated, as a first approximation at least, by a very simple model which postulates that the luckiest, rather than the fittest, are the ones that survive and flourish, and all species are identical,” said Michael Kalyuzhny, a graduate student in theoretical ecology at the Hebrew University of Jerusalem. “Surprisingly, this theory was very successful at explaining the commonness, rarity and number of species in many ecological communities.”

The theory’s successes matter because they help ecologists understand what might be generating patterns in ecosystems. It’s a mystifying fact, for instance, that the majority of tree species in rainforests around the globe are rare. One would expect that the rare species would be outnumbered and ultimately displaced by common species, but instead the rare species persist. Most ecological models can’t mimic this situation. But it happens to be one of neutral theory’s most triumphant successes, implying that the persistence of rarity doesn’t require anything more complicated than randomness and the immigration of new species.

Beyond its usefulness to basic science, understanding large-scale patterns in biodiversity has practical implications. If conservation managers have a more complete understanding of the distribution of rare and common species, they can better handle the ecosystems under their care, said Tak Fung, a postdoc in Chisholm’s lab at the National University of Singapore and the lead author of the new paper.

But a neutral model only works if the static image ecologists see when they pause the simulation is all they are looking for — if they just want a snapshot. If, instead of looking at the model’s general effect, they want to see how it behaves at multiple points over time, the resemblance to reality disintegrates. In the real world, cases like that of Inga marginata, which doubled its numbers in five years, are common. But according to the neutral model, the chances that the tree will double its population from one census to another are around 1 in 101,000, Chisholm calculates — ridiculously small. “We want to know [what’s missing,] what kinds of factors are driving these large fluctuations in species abundances,” Fung said.

Fung and others think that what’s missing could be the effects of environmental changes — a cold snap or heat wave or flood that can dramatically affect the fitness of a species. The idea is that these changes matter in the longer term, with the effects building up and spinning out to affect the evolution of the ecosystem. In the new study, the team focuses on two forests, one in Panama and one in Malaysia. Using census data spanning nearly 30 years, they present two contrasting models of each forest. One model is a neutral model, and the other includes extra variation intended to reflect environmental change.

In the second model, instead of the dice being rolled for every individual in the forest, they’re rolled for each species. If one individual of a given species has a bad year, all the members of that species will have a bad year. And if one individual has a good year, so does the whole species. The process is meant to reflect what might be happening when there’s pervasive environmental change. For example, perhaps a drought hits the forest. A species that thrives in dryness will proliferate, while one that needs lots of water will fail to reproduce. “If you think about what this means in a population of 1,000 individuals,” Chisholm said, “they can all have a good year and all reproduce, and then you get a big increase in population size. Or they can all have a bad year, and a bunch of them die and you get a big decrease.”

The researchers ran each of their simulations thousands of times, to see what the range of outcomes were for each model. (The work kept a computer cluster busy for months.) Then they looked to see whether the real data from the forests — counts of trees painstakingly gathered across many hectares — fell within those ranges.

Both models approximated the mix of rare and common species well, they found. The two models also predicted both the total number of species and the total number of individuals. But only the model with environmental variance could approximate the changes in each species’ numbers over the years.

It’s an exciting finding for Kalyuzhny, who was not involved in this work, as he and his colleagues have found something similar with another model that incorporates environmental change. Before these discoveries, no one knew whether the introduction of extra variation could reproduce the capabilities of neutral theory and also explain fluctuations. Now at least two groups have found that it can.

The findings are part of a larger trend that promises to resolve neutral theory’s problems with predicting how long a species will exist, said Steve Pacala, an ecologist at Princeton University who is interested in neutral theory, although he does not focus on it himself. He notes that neutral theory and other ideas like it aren’t necessarily popular with many ecologists. “Biologists are taught to revere the particular,” he said. “To be a good field biologist, especially a tropical biologist, means to see something in a rainforest and identify what no one else can, or to discover something that is particular and new and beautiful and wonderful. Here you have a bunch of people who try to model the system as though absolutely none of the special qualities of organisms matter, and then get a fair amount of it right. … I think the authors of the paper are really at the vanguard of that area. Although they continue to receive a fair amount of resistance, I think they’re probably right.”

Still, the study leaves some important questions unanswered. Steve Hubbell, an ecologist at the University of California, Los Angeles, who first developed neutral theory, said that the paper and others like it show that taking environmental variance into account is necessary in order to make the theory work over time, at least for some species. Environmental variance seems to be more important in predicting the variation of common species than rare ones, he said, perhaps because the rare species are most sensitive to the kind of randomness that is already baked into neutral theory.

But, he points out, the work does not explain what forms of environmental variation might have caused the change in population sizes. “We don’t learn anything from this paper about the sources of this variance in terms of what’s actually causing it,” he said. “I buy it; I think it’s true. What does it tell me to do next? I don’t know. There are a gazillion and one possible things that could be driving greater variance, and it’s a nontrivial matter to figure out what the data would have to be in order to specify this or that mechanism.”

So while the new models yield better predictions, they haven’t yet descended from their thirty-thousand-foot view of ecology to grapple with the intriguing question of cause and effect. It isn’t clear which perturbations — floods, droughts, plagues of beetles, temperature rise, to name a few of nature’s macabre possibilities — are expanding from their effects on individuals to shape an ecosystem over the long term. And where does the big picture painted by these theories fade into the detailed, messy, intricate world of field biology, where a species might occupy a niche that no other can fill? It’s an open question.

Fung said that his group is considering how to test whether specific environmental changes can create the effects they see in their model. “It’s a tricky question,” he said. “I think we need to be smart about picking the ones we want to focus upon.” Changes in temperature and precipitation are likely candidates, but the team will first need to decipher the relationships between these changes and the numbers used in their model — that is, to determine how rain or heat or storms affect the fitness of different species. In Fung’s view, this work will unfold over the course of the next two decades.

Adding more detail to neutral theory might seem to hamper what it’s best at — the prediction of general patterns across many different ecosystems. But maybe that’s an acceptable trade-off. James O’Dwyer, a theoretical ecologist at the University of Illinois, Urbana-Champaign, who is a co-author of the new paper, suggests that high-level ideas in ecology like neutral theory may be like the ideal gas law, an equation from physics that describes the general properties of gases very well. To make predictions about atypical systems, such as those at low temperatures or high pressures, scientists use another set of more-specific laws. Developing adaptations to neutral theory may be necessary in the same way.

Regardless of which models ultimately turn out to be the most successful, Chisholm said that the question of how environmental changes affect the diversity of species in ecosystems is growing more important all the time. “If environmental variance is going to be increasing in the future, what is the effect on diversity going to be?” he said. “We really need good models of environmental variance if we’re going to make predictions about ecosystems. Our work is a step in that direction.”

This article was reprinted on TheAtlantic.com.