Cell migration is a physical process central to life. Among others, it regulates embryogenesis, tissue regeneration, and tumor growth. Therefore, understanding and controlling cell migration represent fundamental challenges in science. Specifically, the ability of cells to follow stiffness gradients, known as durotaxis, is ubiquitous across most cell types. Even so, certain cells follow positive stiffness gradients while others move along negative gradients. How the physical mechanisms involved in cell migration work to enable a wide range of durotactic responses is still poorly understood. Here, we provide a mechanistic rationale for durotaxis by integrating stochastic clutch models for cell adhesion with an active gel theory of cell migration. We show that positive and negative durotaxis found across cell types are explained by asymmetries in the cell adhesion dynamics. We rationalize durotaxis by asymmetric mechanotransduction in the cell adhesion behavior that further polarizes the intracellular retrograde flow and the protruding velocity at the cell membrane. Our theoretical framework confirms previous experimental observations and explains positive and negative durotaxis. Moreover, we show how durotaxis can be engineered to manipulate cell migration, which has important implications in biology, medicine, and bioengineering.