We consider a class of sparse random matrices which includes the adjacency matrix of the Erdős-Rényi graph G ( N , p ) . We show that if N ε ⩽ N p ⩽ N 1 / 3 - ε then all nontrivial eigenvalues away from 0 have asymptotically Gaussian fluctuations. These fluctuations are governed by a single random variable, which has the interpretation of the total degree of the graph. This extends the result (Huang et al. in Ann Prob 48:916-962, 2020) on the fluctuations of the extreme eigenvalues from N p ⩾ N 2 / 9 + ε down to the optimal scale N p ⩾ N ε . The main technical achievement of our proof is a rigidity bound of accuracy N - 1 / 2 - ε ( N p ) - 1 / 2 for the extreme eigenvalues, which avoids the ( N p ) - 1 -expansions from Erdős et al. (Ann Prob 41:2279-2375, 2013), Huang et al. (2020) and Lee and Schnelli (Prob Theor Rel Fields 171:543-616, 2018). Our result is the last missing piece, added to Erdős et al. (Commun Math Phys 314:587-640, 2012), He (Bulk eigenvalue fluctuations of sparse random matrices. arXiv:1904.07140), Huang et al. (2020) and Lee and Schnelli (2018), of a complete description of the eigenvalue fluctuations of sparse random matrices for N p ⩾ N ε .