Stress affects many brain regions, including the ventral tegmental area (VTA), which is critically involved in reward processing. Excessive stress can reduce reward-seeking behaviors but also exacerbate substance use disorders, two seemingly contradictory outcomes. Recent research has revealed that the VTA is a heterogenous structure with diverse populations of efferents and afferents serving different functions. Stress has correspondingly diverse effects on VTA neuron activity, tending to decrease lateral VTA dopamine (DA) neuron activity, while increasing medial VTA DA and GABA neuron activity. Here we review the differential effects of stress on the activity of these distinct VTA neuron populations and how they contribute to decreases in reward-seeking behavior or increases in drug self-administration.