Small public spaces are the key built environment elements that provide venues for various of activities. However, existing measurements or approaches could not efficiently and effectively quantify how small public spaces are being used. In this paper, we utilized a deep convolutional neural network to quantify the usage of small public spaces through recorded videos as a reliable and robust method to bridge the literature gap. To start with, we deployed photographic devices to record videos that cover the minimum enclosing square of a small public space for a certain period of time, then utilized a deep convolutional neural network to detect people in these videos and converted their location from image-based position to real-world projected coordinates. To validate the accuracy and robustness of the method, we experimented our approach in a residential community in Beijing, and our results confirmed that the usage of small public spaces could be measured and quantified effectively and efficiently.