Theory of selection for variance

The tendency for natural selection to select against variance in offspring number has been discussed before. Indeed, Gillespie has argued that large variance in offspring number could be selected against because adverse outcomes (few or zero offspring) cannot easily be balanced by favorable outcomes (large broods) for individuals of the same species, because the individuals without offspring may not get to try the “offspring lottery” again19,48. Further, Gillespie proposed an approximate mean actual fitness that takes the variance in the offspring number distribution into account (see Equation [6]).

To test whether evolution can explain the emergence of risk sensitive strategies, we generalize the equivalent mean payoff gamble so that there are an infinite number of possible choices, parameterized by the probability χ to obtain the high payoff. We choose this payoff to be 1/χ, so that the mean payoff of any choice will be 1. We will call any of the possible gambles a strategy and denote each strategy by the probability χ. The choice χ = 1 then implies that the agent chooses the safest gamble. In this game, there is no limit on how risky the gamble is, except we do not permit strategies with χ = 0, as they are not normalizable.

Gillespie's fitness estimate w act strongly depends on the strategy choice χ (because it determines the variance in the offspring number distribution) as well as population size (Figure 1A). To illustrate the phenomenon, imagine one risky strategy (χ = 0.01) and 99 non-risky strategies (χ = 1.0). The risky strategy wins on average in one out of hundred gambles and receives a payoff of 100. While the one lucky gamble makes it 100 times fitter than all others, this win happens rarely. However, in the small population of 100 organisms, the risky strategy will have 0.0 payoff and no offspring in 99 out of 100 gambles (generations), it will thus die out more likely. Only in an infinite population will this risky strategy never die out. Therefore, small populations favor, risk averse strategies.

Figure 1 (A) Fitness (w act ) as a function of strategy choice χ and population size according to Gillespie's model Equation (6). Fitness differences between strategies with different χ are far more pronounced in smaller populations. Larger population sizes effectively buffer risky strategies against immediate extinction when the risky strategy does not pay off. See legend for population sizes. We note that the x-axis is log scale. (B) Fixation probability (Π) of a perfectly risk-sensitive strategy (χ = 1) within a uniform background of strategies with choice χ, as a function of χ (solid line). Fixation probabilities were estimated from 100,000 repeated runs, each seeded with a single invading strategy with χ = 1 in a background population of N = 50 resident strategies with strategy χ. The dashed line is Kimura's fixation probability Π(s) = (1 − e−2s)/(1 − e−2Ns) (see, e.g., Ref. 55), where s is the fitness advantage of the invading strategy calculated using Equation (6). Error bars are two standard errors. Full size image

Thus, this theory explains why agents that evolved in small populations show a preference for risk sensitive strategies, whereas agents that evolved in larger populations showed no such strategy preference. We can test the theory directly by measuring the probability Π (the fixation probability) that a perfectly risk-sensitive strategy (χ = 1) can invade (and replace) a homogeneous population consisting of strategies with choice χ ≤ 1. We find that the observed probability of fixation (shown in Figure 1B for a population size of N = 50) agrees qualitatively with the fixation probability calculated using Gillespie's fitness in Kimura's formula (dashed line in Figure 1B), but not quantitatively. Indeed, an effective fitness of about half of Gillespie's estimate reproduces the simulations almost exactly, which corroborates earlier findings22). Since our computational model does otherwise not produce controversial results (see below), we suggest that Equation (3) is quantitatively wrong and therefore does not represent our model.

Evolution in a single population

Evolution is not explained by fixation probabilities alone. In nature we find populations containing a range of mutated strategies that adapt in a heterogeneous background of other mutants. Instead of relying on fixation probabilities alone, we now investigate how in an agent-based model strategies playing the equivalent mean payoff gamble evolve, assuming a non-zero mutation rate. This is particularly important because very risky strategies, in case they win, will have very many offspring in that generation, which all have the chance to mutate.

Each agent in a population is represented by a single probability χ (the agent's inherited gambling strategy), where χ determines the fitness of the agent. Every agent only plays the gamble once in their lifetime, so their fitness is determined by polling a random variable X

exactly once, where p is the probability to receive the corresponding payoff and χ is the agent's strategy. An agent equipped with a strategy χ > 0.5 is considered risk sensitive, whereas an agent with a strategy χ < 0.5 is considered risk-prone. All else being equal, we expect that evolution should not prefer any strategy over another, because the mean payoff of a species (individuals with the same χ) should be the same regardless of χ.

If population size does not have a significant effect on the evolution of risk sensitivity, we would expect the strategy preference of any individual to drift neutrally, so that at the end of the evolutionary run (generation 950) the expected mean population strategy is (the mean of a uniformly distributed random variable constrained between zero and one). Instead, we observe in Figure 2A that the mean χ converges to 0.6941 ± 0.0139 (mean ± two standard errors).

Figure 2 Strategy evolution with a fixed population size of 100 individuals and a mutation rate of 1%. (A) Mean strategy on the line of descent at generation 950 over 1,000 replicate runs. Measurements were taken after selection happened, hence the value for generation 1 is not 0.5. The agents on the line of decent show a preference for risk sensitive strategy. The dotted line indicates the expected value 0.5 for unbiased evolution, i.e., no strategy preference. (B) The probability distribution of χ at generation 950 of the dominant strategy across 1,000 replicate runs. This is identical to the distribution of strategies within the population at generation 950 (Figure S2), showing that there is no considerable difference between line of descent and population averages. The agents evolve a significant preference for risk sensitive strategies by generation 950 (Wilcoxon rank sum test P = 7.795410−22 between this distribution and a uniform random distribution). Full size image

Similarly, we would expect that if the strategy drifts neutrally, we should observe χ to be distributed in a uniform manner at the end of the evolutionary runs. Instead, for a population size of N = 100, we observe in Figure 2B a distribution that departs significantly from uniformity (Wilcoxon rank sum test P = 7.795410−22 between this distribution and a uniform random distribution). This result suggests that population size plays a critical role in shaping what strategies evolve in the agent population.

To further explore the effect of population size on evolved strategy preference, we ran the evolutionary simulation with different fixed population sizes. Figure 3 demonstrates that the final evolved strategy depends strongly on the population size. These results highlight that agents in smaller populations prefer risk sensitive strategies that receive a lower payoff but with higher reliability. In contrast, agents in larger populations do not show a preference for risk sensitivity nor risk-seeking strategies and converge on because all strategies perform roughly the same in large populations, that is, the χ of individual strategies drifts neutrally.

Figure 3 Mean strategy at the end of 1,000 evolutionary runs as a function of population size. Agents in smaller populations (e.g., 50 and 100) demonstrate a clear preference for risk sensitive strategies. In contrast, agents in larger populations (e.g., size 5,000 and 10,000) display only weak risk sensitivity or no preference. Error bars are two standard errors over 1,000 replicates. The dotted line indicates the expected mean value for unbiased choice, i.e., no strategy preference. Full size image

Throughout evolutionary history, humans have experienced at least two population bottlenecks that reduced the human population to as few as 1,000 individuals50,51. However, a population size of 1,000 individuals is unlikely to be small enough to evolve risk sensitive behavior as a dominant strategy in the population19,48. Instead, a more likely explanation is that humans have lived in groups of about 150 individuals throughout their evolutionary history52,53, which plausibly could have been a small enough effective population size for risk sensitivity to have evolved.

Evolution in groups

In the previous section, we demonstrated that agents in small populations evolve a preference for strategies with low variance in their payoff distribution, i.e., risk sensitivity. The group size for humans throughout evolutionary history has been proposed to be around 150 individuals52,53, which suggests that evolving in such small groups could have been the reason behind the evolution of human risk sensitivity. However, a small group size and a small population size are two different things. While humans might have lived in small groups of 150 individuals, the total population size of humans has been much larger and were only at times as low as 1,000 individuals50,51. Even though selection may occur within groups of about 150, individuals likely migrated between groups. Migration could have caused selection to effectively act on much larger groups (or even the entire human population) negating the selection for a variance effect.

We can simulate such an environment using an island-based evolutionary model (see Methods), in which individuals live in groups (the “islands”) that randomly exchange individuals with each other via migration. For example, we can run 1,000 replicate evolutionary experiments with 128 groups of 128 individuals each, with varying migration rates. In this configuration the total population size is 16,384 individuals, which according to Figure 3 should result in agents evolving no strategy preference. Figure 4 shows that regardless of the migration rate, the group size and not the total population size determines whether agents evolve risk sensitive strategies. This result suggests that even with migration between groups, the effective population size that selection acts on is determined by the group size and not the total population size. While this result might seem surprising, one has to take into account that selection happens for each group separately and these virtual agents only compete against agents within their group and not against everyone in the entire population, which explains why the migration rate does not affect the evolution of risk aversion in this model.

Figure 4 Mean strategy on the line of descent at generation 950 as a function of the migration rate in an island model genetic algorithm with 128 groups with 128 members in each group. Regardless of the migration rate, it is the group size and not the total population size that determines if the agents evolve risk sensitive strategies. Error bars indicate two standard errors over 1,000 replicates. The dotted line indicates the expected value for unbiased choice, i.e., no strategy preference. A migration rate of 0.5 implies that half of the agents in each group migrate every generation. Full size image

When we change the size of the groups but fix the total population size (i.e., increase the group size and reduce the number of groups) while keeping the migration rate at a constant 0.1, we again observe that the group size critically determines the preferred evolved agent strategies (Figure 5). Risk sensitive strategies are preferred in smaller groups and no strategy is preferred in larger groups. This result recapitulates the results from Figure 3, which shows that the preference for strategies with low payoff variation (i.e., risk sensitivity) depends on the effective population size.

Figure 5 Mean strategy on the line of descent at generation 950 as a function of the ratio between group size and number of groups. Group size critically determines the preferred evolved agent strategies, where risk sensitive strategies are preferred in smaller groups and no strategy is preferred in larger groups. The x-axis tick labels are formatted as . Error bars indicate two standard errors over 1,000 replicates. The dotted line indicates the expected value for unbiased choice, i.e., no strategy preference. Full size image

Relative value of the gamble

Another way to alter risk sensitivity in humans is by changing the relative value of the payoff42. When the gamble is about small amounts of money (i.e., “peanuts” gambles or hypothetical money), humans tend to be less risk averse, whereas raising the relative value of the gamble increases risk aversion. In our evolutionary simulation, agents play the gamble a single time and the payoff they receive is their only source of fitness. This constraint effectively turns the gamble into a life or death situation, similar to a game with extraordinarily high stakes.

To simulate lower-stakes gambles, we add a baseline payoff (β) to the payoff so that the fitness of the agent becomes

where p is the probability to receive the corresponding payoff and χ is the agent's strategy. Typical gambles humans partake in fall either in the loss or in the gain domain. In biological systems, on the other hand, organisms accumulate resources in order to ultimately produce offspring. The “gambles” these organisms undertake will influence the number of offspring, which will be positive or zero. Thus, we cannot differentiate between losses or gains in the same way people conceive a gamble for money. Therefore, gains and losses must be considered relative to fitness.

When we run the evolutionary simulation with a population size N = 100 for various values of β, we observe that the larger the baseline β becomes, the more often strategies return to an unbiased choice (Figure 6). This result is expected because fitness differences only matter if their relative impact is larger than 54,55. Thus, risk sensitive strategies will only be selected for when the outcome of the gamble represents a significant portion of the individual's fitness when taking the population size into account.

Figure 6 Mean strategy on the line of descent at generation 950 depending on the additional payoff (β). The larger the additional payoff β becomes, the more often strategies return to an unbiased choice. Error bars indicate two standard errors over 1,000 replicates. The dotted line indicates the expected value for unbiased choice, i.e., no strategy preference. Full size image

Repetition of the gamble

Thus far, we have only investigated one-time gambles. What happens when the agents engage in the same gamble multiple times during their lifetime? On average, repeating the gamble reduces the variance in the overall payoff the agents receive and if games are played infinitely, then the payoffs will converge to the same mean. In this experiment, we do not consider situations where agents can change their behavior based on previous experiences56, but rather focus on unconditional responses. We observe that the agents no longer evolve a preference for risk sensitivity if the gamble is repeated several times in a lifetime (Figure 7). At the same time, the effect of repetition depends strongly on the population size, such that smaller populations still evolve risk sensitive strategies with as many as 10 repetitions of the gamble. Therefore, a preference for risk sensitivity will only evolve for those gambles that are encountered a few times during an individual's lifetime.