Limitations of the inference of the distribution of fitness effects of new mutations in partially selfing populations with linkage.

The accurate estimation of the distribution of fitness effects (DFE) of new mutations is critical for population genetic inference but remains a challenging task. While various methods have been developed for DFE inference using the site frequency spectrum of putatively neutral and selected sites, their applicability in species with diverse life history traits and complex demographic scenarios is not well understood. Selfing is common among eukaryotic species and can lead to decreased effective recombination rates in such populations, increasing the effects of selection at linked sites, including interference between selected alleles. We employ forward simulations to investigate the limitations of current DFE estimation approaches in the presence of selfing and linked effects of selection. We find that distortions of the site frequency spectrum due to Hill-Robertson interference in highly selfing populations lead to mis-inference of the deleterious DFE of new mutations. While accounting for the decrease in the effective population size due to linked effects of selection largely accounts for the observed bias in populations with moderate levels of selfing, this correction is unable to accurately estimate the DFE in highly selfing populations. In addition, the presence of cryptic population structure and uneven sampling across subpopulations leads to the false inference of a deleterious DFE skewed towards effectively neutral/mildly deleterious mutations. Finally, the proportion of adaptive substitutions estimated at high rates of selfing is substantially overestimated. Our results clarify the parameter space where current DFE methods might be problematic and where they remain robust in the presence of selfing and other model violations, such as departures from semidominance, population structure, and uneven sampling. Our observations apply broadly to species and genomic regions with little/no recombination.