Non-dormant seeds are continuously aging and deteriorating during storage, leading to declining seed vigor, which is a challenge for the rice seed industry. Improving the storability of seeds is of great significance to ensure the quality of rice and national food security. Through a set of chromosome segment substitution lines population constructed using japonica rice NIP as donor parent and indica rice ZS97 as recurrent parent, we performed seed storability QTL analysis and selected four non-storable NILs to further investigate the storability regulatory mechanisms underlying it. The seeds were divided into four tissues, which were the embryo, endosperm, aleurone layer, and hull, and tissue-specific transcriptome and metabolome analyses were performed on them. By exploring the common differentially expressed genes and differentially accumulated metabolites, as well as the KEGG pathway of the four non-storable NILs, we revealed that the phenylpropanoid biosynthesis pathway and diterpenoid biosynthesis pathway played a central role in regulating seed storability. Integrated analysis pinpointed 12 candidate genes that may take part in seed storability. The comprehensive analysis disclosed the divergent and synergistic effect of different seed tissues in the regulation of rice storability.