The Drive for Muscularity Scale (DMS) is a commonly used measure used to assess the pursuit of muscularity. However, the factor structure of this measure has yet to be confirmed in a sample of sexual minority women. Moreover, the invariance of this measure across gender has also yet to be explored. The aim of the present study was, therefore, to conduct a confirmatory factor analysis (CFA) of the DMS in samples of both cisgender sexual minority men and women, and subsequently evaluate the measurement invariance by gender. The sample consisted of 962 cisgender sexual minority young adult men ( N = 479) and women ( N = 483). A series of CFAs were conducted, assessing both the one-factor and two-factor solutions of the DMS, with and without the inclusion of item 10 ("I think about taking anabolic steroids"). Across cisgender sexual minority young adult men and women, the 14-item two-factor solution demonstrated most appropriate fit, although the 15-item two-factor solution was also adequate among only women. Measurement invariance analyses indicated that the 14-item two-factor DMS can be used in samples of both cisgender sexual minority men and women. The present study was novel in exploring the factor structure of the DMS in sexual minority women and measurement invariance by gender; however, future research is needed to further corroborate these findings and assess measurement invariance by sexual orientation and race.