Recurrent selection has not been extensively applied in rice breeding practices due to lack of male sterility genes. Recently, a male sterile line (named as 'Jiabuyu') controlled by a novel single dominant gene was found, which provides an ideal tool for rice recurrent selection breeding. Here, two different kinds of recurrent selection populations facilitated for outcrossing by the dominant male sterile line 'Jiabuyu' were developed, that included one population using 31 abiotic stress tolerance introgression lines with common recipient parent as founders, and the other one using 25 popular restorers lines applied in hybrid breeding as founders. After two cycles of recurrent selection through natural outcrossing, the seeds from male fertile progeny plants were screened for higher grain yield under normal irrigated, drought, and salt-stressed natural field conditions. Finally, we identified 11 promising high-yielding lines under irrigated conditions, 12 drought-tolerant and 12 salt-tolerant lines. Among them, one line gave higher grain yield across all three conditions, three lines yielded high in both irrigated and drought conditions and another three lines gave high yields in both drought and salt-stressed conditions. The present study was a commendable attempt at utilizing recurrent selection population facilitated by dominant male sterile line to improve rice complex traits, which provided valuable lessons for other rice breeders. The developed lines are promising to be rice varieties with high yield, drought and/or salinity tolerances.