Forest canopies, an essential part of forest ecosystems, are among the most highly threatened terrestrial habitats. Mountains provide ideal conditions for studying the variation in community structure with elevations. Spiders are one of the most abundant predators of arthropods in terrestrial ecosystems and can have extremely important collective effects on forest ecosystems. How the diversity and composition of canopy spider communities respond to elevation changes in temperate forests remains poorly understood. In this study, we collected canopy spiders from four elevation sites (800 m, 1100 m, 1400 m, and 1700 m) on Changbai Mountain using the fogging method in August 2016. With the methods of ANOVA analysis, transformation-based redundancy analysis, and random forest analysis, we explored the responses of canopy spider communities to elevation. In total, 8826 spiders comprising 81 species were identified and the most abundant families were Thomisidae, Clubionidae, Linyphiidae, and Theridiidae (77.29% of total individuals). Species richness decreased whereas evenness increased with increasing elevation, indicating that elevation has an important impact on community structure. The pattern of absolute abundance was hump shaped with increasing elevation. We found that the community compositions at the three taxonomic levels (species, family, and guild) along the elevation gradient were obviously altered and the variation in community composition was higher at low-elevation sites than at high-elevation sites. There were 19 common species (23.46%) among the four elevations. Regression and RDA results showed that vegetation variables contributed to the variation in the diversity and composition of canopy spiders. Furthermore, the influence of factors would be weakened with the taxonomic level increasing. Therefore, our findings greatly highlight the important role of vegetation in the diversity and composition of canopy spiders and the influence is closely related to the taxonomic level.