The evolution of strong reproductive isolation

Felsenstein distinguished two ways by which selection can directly strengthen isolation. First, a modifier that strengthens prezygotic isolation can be favored everywhere. This fits with the traditional view of reinforcement as an adaptation to reduce deleterious hybridization by strengthening assortative mating. Second, selection can favor association between different incompatibilities, despite recombination. We generalize this “two allele” model to follow associations among any number of incompatibilities, which may include both assortment and hybrid inviability. Our key argument is that this process, of coupling between incompatibilities, may be quite different from the usual view of reinforcement: strong isolation can evolve through the coupling of any kind of incompatibility, whether prezygotic or postzygotic. Single locus incompatibilities become coupled because associations between them increase the variance in compatibility, which in turn increases mean fitness if there is positive epistasis. Multiple incompatibilities, each maintained by epistasis, can become coupled in the same way. In contrast, a single-locus incompatibility can become coupled with loci that reduce the viability of haploid hybrids because this reduces harmful recombination. We obtain simple approximations for the limits of tight linkage, and strong assortment, and show how assortment alleles can invade through associations with other components of reproductive isolation.