We use a three‐pronged theoretical approach to investigate how these different mechanisms could interact, and their relative importance (Frank, 1996 ). We first build an analytical model of a specified symbiont life cycle in which we can tease apart the separate causal influences of relatedness and transmission mode. This allows us to test which mechanism plays the larger causal role in the evolution of cooperation. Then, by expressing relatedness in terms of symbiont transmission mode and bottlenecking between symbiont generations (“closing” the model), we allow transmission mode to influence relatedness. This allows us to partition the influence of transmission mode per se, and via its effect on relatedness (Cooper et al., 2018 ). Finally, we test the robustness of our conclusions with an individual‐based simulation. This simulation allows us to relax several assumptions, including that mutations are of small size, and that the trait value for cooperation does not influence relatedness. Our simulation also allows us to investigate whether evolutionary branching can occur, as has been observed in the early stages of experimentally evolved mutualisms (Harcombe et al., 2018 ).

Both of these mechanisms, “transmission” and “relatedness”, could operate, and both their relative importance and the extent to which one influences the other remain unclear. The empirical observation that vertically transmitted symbionts provide greater benefits to their hosts could be explained by either mechanism, or by both acting simultaneously. Theoretical studies tend to make simplifying assumptions that allow them to focus on just one of these mechanisms (Frank, 1996 ). For example, some of the studies that emphasize transmission mode assume that hosts can only be infected by one strain of symbiont at a time, ignoring the possibility for conflict between symbionts within a host (Yamamura, 1993 , 1996 ). Similarly, models that examine the influence of variable relatedness do not usually explicitly model horizontal and vertical transmission (Frank, 1994 , 2010 ). In nature, both mechanisms are likely to occur, and we have a poor understanding of the consequences. For example, would they have distinct and different influences, or would they interact; would one drive the other, or would one tend to dominate?

Two different mechanisms have been given for why the mode of symbiont transmission matters (Frank, 1996 ). One mechanism is that if symbiont offspring are likely to be transmitted to host offspring, then symbionts benefit when the host has more offspring (Ewald, 1987 ; Yamamura, 1993 , 1996 ; Ferdy & Godelle, 2005 ). In this “transmission” scenario, it is vertical transmission per se that selects for higher levels of symbiont cooperation, through aligning the fitness interests of hosts and symbionts—vertical transmission makes symbionts more dependent upon their hosts. The other mechanism is that the transmission route determines the genetic diversity or relatedness between the symbionts and that this determines selection for cooperation (Hamilton, 1964 ; Frank, 1994 , 1996 ; Herre et al., 1999 ; West et al., 2002 ; Foster & Wenseleers, 2006 ). Greater horizontal transmission will lead to a lower relatedness between symbionts. As relatedness between symbionts goes down, this can favour symbionts who avoided the cost of helping their hosts, but could still benefit from the benefits provided to the hosts by other symbionts. In this “relatedness” scenario, transmission mode matters, but it does so through its influence on relatedness—vertical transmission reduces conflict between symbionts.

There is considerable variation in the benefit that hosts gain from their symbionts. In some cases, hosts are completely dependent upon their symbionts. For example, aphids cannot survive or reproduce without Buchnera symbionts, which provide essential amino acids (Buchner, 1965 ; Douglas, 1998 ). In other cases, symbionts appear to provide relatively minor benefits. For example, the removal of Chlorella symbionts from Paramecium bursaria leads to just a reduction in growth rate, and only under certain conditions (Karakashian, 1963 ; Lowe et al., 2016 ). Empirical studies have suggested that the way in which symbionts are transmitted between hosts plays an important role in explaining this variation (Bull et al., 1991 ; Bull & Molineux, 1992 ; Herre, 1995 ; Messenger et al., 1999 ; Sachs & Wilcox, 2006 ; Fisher et al., 2017 ). Specifically, that vertical transmission, where hosts transmit symbionts to their offspring, selects for more cooperative symbionts than horizontal transmission, where symbionts can leave their host and be transmitted to other individuals in the population. Symbionts which are more cooperative could in turn provide greater benefits to their hosts, by investing more of their resources into functions which benefit their hosts or by refraining from overexploiting their hosts’ resources (Frank, 1994 , 1996 ).

2 MODELS AND RESULTS

2.1 Assumptions and model life cycle We assume a mutualism in which symbionts live inside hosts and potentially provide them with some benefit. We assume that the symbionts cannot survive long enough to reproduce outside the hosts, and so they are obligately dependent on the hosts. We assume that there is an infinite population of hosts with nonoverlapping generations and that there is no host population structure. We assume that the cooperative symbiont trait x denotes the amount of resources contributed towards a service which benefits the host, but which does not directly benefit the symbiont. For example, this trait could be the production of a key nutrient that the host needs. We assume that hosts with more cooperative symbionts are more likely to survive to reproductive maturity and are more likely to produce more offspring after reaching reproductive maturity. Therefore, we assume that symbiont cooperation can benefit both host survival and host fecundity, according to the functions s(x g ) and f(x g ), respectively, where x g refers to the mean investment into cooperation of all of the symbionts inside a focal host. We use mean, and not total, symbiont investment into cooperation, for the sake of simplicity, and to be consistent with previous work (Frank, 1994, 1996). We also assume that this trait is costly to the symbiont, by assuming that a focal symbiont's growth rate inside a host depends negatively on its investment into cooperation, according to the expression , where x i is a focal symbiont's investment into cooperation. We assume that a symbiont can potentially transmit offspring to future generations via two routes, vertical or horizontal: vertical transmission occurs when a symbiont's offspring remain in their host and are passed on to the host's offspring; horizontal transmission is when a symbiont's offspring can infect the offspring of any host in the population. We assume that increased host survival increases the transmission opportunities for horizontally transmitting symbionts, and so we weight the horizontal component of symbiont fitness by a focal symbiont's host's relative survival, , where is the mean level of symbiont cooperation in the population as a whole. We assume that both host survival and host fecundity per unit time increase the transmission of vertically transmitting symbionts, and so we weight the vertical component of symbiont fitness by . λ to capture the relative likelihood of horizontal (λ) compared to vertical (1‐ λ) transmission. λ could be influenced by a number of different biological factors, including if hosts are more likely to reject symbionts from one route than the other, or if one mode of transmission involves higher symbiont mortality. The fitness of a focal symbiont is then: (1) Finally, we use a parameterto capture the relative likelihood of horizontal () compared to vertical (1‐) transmission.could be influenced by a number of different biological factors, including if hosts are more likely to reject symbionts from one route than the other, or if one mode of transmission involves higher symbiont mortality. The fitness of a focal symbiont is then: This fitness equation sets up a trade‐off similar to other models of cooperative traits (Frank, 1994, 2010). Figures were produced using Wolfram Mathematica 11.3 (Harrower & Brewer, 2003; Wang, 2016).

2.2 Equilibrium analysis x*) which, if adopted by all symbionts in the population, could not be beaten by any alternative value of x, which is termed an evolutionarily stable strategy (ESS). We used a neighbour‐modulated fitness approach to obtain the inclusive fitness effect, ∆ IF , of small changes in the trait value for cooperation on the inclusive fitness of a focal individual, assuming the limit of weak selection (Taylor & Frank, 1996 (2a) We are interested in the level of investment into cooperation () which, if adopted by all symbionts in the population, could not be beaten by any alternative value of, which is termed an evolutionarily stable strategy (ESS). We used a neighbour‐modulated fitness approach to obtain the inclusive fitness effect,, of small changes in the trait value for cooperation on the inclusive fitness of a focal individual, assuming the limit of weak selection (Taylor & Frank,): x*, evaluating all derivatives at (Maynard Smith & Price, 1973 and , where s > 0 and f > 0, and so arrive at: (2b) f or s indicate that host fecundity or survival respectively increases more quickly with symbiont cooperation, and R is the whole‐group relatedness coefficient (Taylor & Frank, 1996 2000 We solved △IF = 0 for*, evaluating all derivatives at(Maynard Smith & Price,). To allow for a wide range of relationships between symbiont cooperation and host survival or fecundity, we assume thatand, where> 0 and> 0, and so arrive at:where higher values oforindicate that host fecundity or survival respectively increases more quickly with symbiont cooperation, andis the whole‐group relatedness coefficient (Taylor & Frank,; Pepper,). Equation 2b allows us to see the different effects of changes in cooperation (x*) on the inclusive fitness of a focal individual. The first term in Equation 2b is the cost of cooperation (x*), which reflects reduced symbiont competitiveness within a host. The second term in Equation 2b is the benefit of cooperation that goes to the other symbionts sharing the focal symbiont's host, weighted by the genetic relatedness between the focal symbiont and its neighbours (R). This benefit stems from the fact that more cooperative (higher x*) groups of symbionts will have hosts that live longer (in a way that scales with s) and have more offspring (in a way that scales with f). , we find solutions which are local maxima, and hence candidate ESSs, over the relevant parameter space(0 ≤ R ≤ 1, 0 ≤ λ ≤ 1), which we denote (Maynard Smith & Price, 1973 1996 2007 2014 2015 (3) By taking the second derivative, we find solutions which are local maxima, and hence candidate ESSs, over the relevant parameter space(0 ≤≤ 1, 0 ≤≤ 1), which we denote(Maynard Smith & Price,; Taylor & Frank,; Otto & Day,; Lehmann & Rousset,; Biernaskie & West,): We found that both relatedness and transmission mode influenced the final level of cooperation in this model (Figure 1). Relatedness increases cooperation because it increases the extent to which the benefits of cooperation go to genetic relatives of the actor. This is reflected by an increased weighting of the second term in Equation 2b, resulting in a higher level of cooperation (x*) when fitness is at equilibrium. Vertical transmission increases cooperation because higher levels of vertical transmission increase the extent to which host fecundity can benefit symbionts (Equation 1). This is reflected in Equation 2b by the fact that vertical transmission (lower λ) increases the f(1–λ) component of the group symbiont benefit (second term of Equation 2b). These findings are consistent with previous work that looked just at transmission mode or just at relatedness (Yamamura, 1993; Frank, 1994). Figure 1 Open in figure viewer PowerPoint s = 0.5, f = 2) Both transmission mode and relatedness influenced the final level of cooperation that emerged (Equation 3) . In (a), host survival and fecundity both increase in the same way with symbiont cooperation (s = f = 1). In (b), host fecundity increases more quickly with symbiont cooperation than host survival does (= 0.5,= 2)

2.3 Transmission or relatedness: open model At this stage, we are interested in asking two different questions of our model. The first question is whether relatedness or transmission mode plays the larger role in determining cooperation. To answer this question, we keep relatedness as an open parameter in our model, allowing us to examine the separate causal influences of relatedness (R) and transmission mode (λ). However, in reality, these factors are not independent, since transmission mode can determine relatedness (Taylor, 1992; Frank, 1996; Cooper et al., 2018). We can capture this by “closing” the model and expressing relatedness in terms of demographic parameters. Closing the model allows us to ask our second question of why transmission mode influences cooperation: primarily through its direct influence on cooperation per se, or primarily through its influence on relatedness? To start with, we keep relatedness as an open parameter. Both relatedness and transmission mode influence the equilibrium level of cooperation (Equation 3). For the parameters chosen in Figure 1, it appears that relatedness plays a larger role than transmission mode, in the sense that small changes to relatedness influence the equilibrium level of cooperation more than small changes in transmission mode do (Figure 1). To extend this comparison over all of the potential parameter space, we compared the marginal effect of changes in transmission mode (λ) or relatedness (R) on the equilibrium level of cooperation. We calculated the marginal effects by taking the differential of the equilibrium level of cooperation with respect to either relatedness ( ) or transmission mode ( ). The first of these differentials ( reflects the alignment of fitness interests between symbionts within a host—to what extent should more highly related groups of symbionts cooperate more? The second of these differentials ( ) reflects the alignment of fitness interests between a host and its symbionts—to what extent does increased vertical transmission favour a host's symbionts to cooperate more? By comparing the value of the two differentials, we can determine whether relatedness or transmission mode has a larger influence on the equilibrium level of cooperation. In Appendix 1, we show that, for most of the possible parameter space, relatedness (R) plays a bigger role than transmission mode (λ) in determining the final level of cooperation (Figure 2). Specifically, transmission mode only plays a larger role if three conditions are all met: (a) horizontal transmission dominates (λ > 0.75); (b) host fecundity accelerates substantially faster with symbiont cooperation than host survival (f > 4s); and (c) relatedness is neither maximal nor minimal (0 < R < 1; Figure 2). Figure 2 Open in figure viewer PowerPoint R) usually had a larger influence on the final level of cooperation than transmission mode (λ) did. In the orange regions plotted, relatedness had a larger influence than transmission mode, whereas in the white regions, transmission mode had a larger influence than relatedness. Transmission mode only had a larger influence when transmission was mostly horizontal (λ > 0.75) and when host fecundity increased more rapidly with symbiont cooperation than host survival did ( ) In the first analytical model (Equation 3) , relatedness () usually had a larger influence on the final level of cooperation than transmission mode () did. In the orange regions plotted, relatedness had a larger influence than transmission mode, whereas in the white regions, transmission mode had a larger influence than relatedness. Transmission mode only had a larger influence when transmission was mostly horizontal (> 0.75) and when host fecundity increased more rapidly with symbiont cooperation than host survival did (

2.4 Transmission and relatedness: closed model et al., 2018 k h symbionts and that vertically infected hosts are infected by k v symbionts. In Appendix (4) k h and k v give the horizontal and vertical bottleneck sizes, respectively, and λ gives the fraction of host offspring that are infected horizontally. Our next step is to “close” the model by expressing relatedness in terms of demographic parameters (Cooper). We assume that hosts infected by symbionts horizontally are infected bysymbionts and that vertically infected hosts are infected bysymbionts. In Appendix 2 , we show that whole‐group relatedness can now be expressed as:whereandgive the horizontal and vertical bottleneck sizes, respectively, andgives the fraction of host offspring that are infected horizontally. Relatedness depends on the extent to which transmission is vertical or horizontal. Under full horizontal transmission (λ = 1), Equation 4 simplifies to , whereas under full vertical transmission, Equation 4 simplifies to 1 (full relatedness). This occurs because horizontal transmission “resets” relatedness by enforcing complete mixing of unrelated symbionts, whereas vertical transmission allows relatedness to increase each generation, since symbionts interact only within a local group. k h = k v =k) to arrive at: (5) Next, we further simplify Equation 4 by assuming that horizontally and vertically transmitting symbionts experience the same bottleneck size (=k) to arrive at: ; Equation : (6) By substituting our expression for relatedness (Equation 5 ) into our expression for the equilibrium level of cooperation (; Equation 3 ), we arrive at a new expression for the equilibrium level of cooperation, which we denote We then compared the extent to which transmission mode influences cooperation via its direct influence and via its influence on relatedness. To do this, we first calculated, as before, the marginal effect of changes in transmission mode on the equilibrium level of cooperation for the model with relatedness left open ( ). Then, we calculated the total effect of changes in transmission mode on the equilibrium level of cooperation, by taking the differential of the expression for equilibrium cooperation after the model has been closed ( ). These two partial derivatives represent, respectively, the influence of transmission mode via its direct influence and the total influence of transmission mode via both influences. We isolate the effect of transmission mode via its influence on R by subtracting the first partial derivative ( ) from the second ( ). By comparing these derivatives, we can then test whether transmission mode matters mostly because it aligns the interests of symbionts sharing a host (by increasing relatedness) or mostly by aligning the interests of symbionts and hosts. In Appendix 3, we show that transmission mode always had a larger influence via its influence on relatedness than via its direct influence , unless: (i) symbiont cooperation increases host fecundity faster than it increases host survival (f > s); (ii) and transmission is mostly horizontal (λ > 0.5; Figure 3). Figure 3 Open in figure viewer PowerPoint R when transmission was mostly vertical (λ < 0.5) or when host survival increased with symbiont cooperation more quickly than host fecundity did (f < s). The dark line plots the point at which transmission mode influences cooperation equally through both routes. For this plot, s = 1 In the second analytical model (Equation 3) , transmission mode influenced cooperation primarily through its influence on relatedness. Transmission mode always influenced cooperation more viawhen transmission was mostly vertical (< 0.5) or when host survival increased with symbiont cooperation more quickly than host fecundity did (< s). The dark line plots the point at which transmission mode influences cooperation equally through both routes. For this plot,= 1 Our closed model highlights how focusing just on transmission mode could lead to misleading predictions about the level of cooperation. Equation 5 shows that if transmission is mostly vertical (low λ), then relatedness will always be high, because the λ(k‐1) term will be small. However, if transmission is mostly horizontal (high λ), then relatedness can either be high or low, depending on the degree of bottlenecking (the value of k) (Equation 5). Consequently, if transmission is mostly horizontal, then focusing just on transmission mode erroneously predicts that a low level of cooperation will evolve, when in fact high levels of cooperation can sometimes evolve (Equation 6).