Counting is an evolutionarily recent cultural invention of the human species. In order for humans to have conceived of counting in the first place, certain representational and logical abilities must have already been in place. The focus of this review is the origins and nature of those fundamental mechanisms that promoted the emergence of the human number concept. Five claims are presented that support an evolutionary view of numerical development: 1) number is an abstract concept with an innate basis in humans, 2) maturational processes constrain the development of humans' numerical representations between infancy and adulthood, 3) there is evolutionary continuity in the neural processes of numerical cognition in primates, 4) primitive logical abilities support verbal counting development in humans, and 5) primitive neural processes provide the foundation for symbolic numerical development in the human brain. We support these claims by examining current evidence from animal cognition, child development, and human brain function. The data show that at the basis of human numerical concepts are primitive perceptual and logical mechanisms that have evolutionary homologs in other primates and form the basis of numerical development in the human brain. In the final section of the review, we discuss some hypotheses for what makes human numerical reasoning unique by drawing on evidence from human and non-human primate neuroimaging research.