Our study demonstrates RHR more than 70 bpm present significant risk effect on cognitive decline, and this relationship is modified by PA. Elder population with physical inactivity and higher RHR should be paid more attention to prevent cognitive decline.