An obvious defect of extreme learning machine (ELM) is that its prediction performance is sensitive to the random initialization of input-layer weights and hidden-layer biases. To make ELM insensitive to random initialization, GPRELM adopts the simple an effective strategy of integrating Gaussian process regression into ELM. However, there is a serious overfitting problem in kernel-based GPRELM ( k GPRELM). In this paper, we investigate the theoretical reasons for the overfitting of k GPRELM and further propose a correlation-based GPRELM ( c GPRELM), which uses a correlation coefficient to measure the similarity between two different hidden-layer output vectors. c GPRELM reduces the likelihood that the covariance matrix becomes an identity matrix when the number of hidden-layer nodes is increased, effectively controlling overfitting. Furthermore, c GPRELM works well for improper initialization intervals where ELM and k GPRELM fail to provide good predictions. The experimental results on real classification and regression data sets demonstrate the feasibility and superiority of c GPRELM, as it not only achieves better generalization performance but also has a lower computational complexity.