Allostery plays a crucial role in regulating protein activity, making it a highly sought-after target in drug development. One of the major challenges in allosteric drug research is the identification of allosteric sites. In recent years, many computational models have been developed for accurate allosteric site prediction. Most of these models focus on designing a general rule that can be applied to pockets of proteins from various families. In this study, we present a new approach using the concept of Learning to Rank (LTR). The LTR model ranks pockets based on their relevance to allosteric sites, that is, how well a pocket meets the characteristics of known allosteric sites. After the training and validation on two datasets, the Allosteric Database (ASD) and CASBench, the LTR model was able to rank an allosteric pocket in the top three positions for 83.6% and 80.5% of test proteins, respectively. The model outperforms other common machine learning models with higher F1 scores (0.662 in ASD and 0.608 in CASBench) and Matthews correlation coefficients (0.645 in ASD and 0.589 in CASBench). The trained model is available on the PASSer platform (https://passer.smu.edu) to aid in drug discovery research.