The spread of synthetic gene drives is often discussed in the context of panmictic populations connected by gene flow and described with simple deterministic models. Under such assumptions, an entire species could be altered by releasing a single individual carrying an invasive gene drive, such as a standard homing drive. While this remains a theoretical possibility, gene drive spread in natural populations is more complex and merits a more realistic assessment. The fate of any gene drive released in a population would be inextricably linked to the population's ecology. Given the uncertainty often involved in ecological assessment of natural populations, understanding the sensitivity of gene drive spread to important ecological factors is critical. Here we review how different forms of density dependence, spatial heterogeneity, and mating behaviors can impact the spread of self-sustaining gene drives. We highlight specific aspects of gene drive dynamics and the target populations that need further research.