Effect of assortative mating and sexual selection on polygenic barriers to gene flow

Assortative mating and sexual selection are widespread in nature and can play an important role in speciation by facilitating the buildup and maintenance of reproductive isolation (RI). However, their contribution to genome-wide suppression of gene flow during RI is rarely quantified. Here, we consider a polygenic “magic” trait that is divergently selected across two populations connected by migration, while also serving as the basis of assortative mating, thus generating sexual selection on one or both sexes. We obtain theoretical predictions for divergence at individual trait loci by assuming that the effect of all other loci on any locus can be encapsulated via an effective migration rate, which bears a simple relationship to measurable fitness components of migrants and various early-generation hybrids. Our analysis clarifies how “tipping points” (characterized by an abrupt collapse of adaptive divergence) arise, and when assortative mating can shift the critical level of migration beyond which divergence collapses. We quantify the relative contributions of viability and sexual selection to genome-wide barriers to gene flow and discuss how these depend on existing divergence levels. Our results suggest that effective migration rates provide a useful way of understanding genomic divergence, even in scenarios involving multiple, interacting mechanisms of RI.