Self-efficacy for medication use not only mediated the relationship between general harm beliefs about medication and medication adherence, but moderated it negatively. The findings of this study indicate an opportunity to improve the prognosis of elderly Chinese patients with type 2 diabetes through improved medication adherence by strengthening factors such as self-efficacy for appropriate medication use and general harm beliefs about medication.