Due to an increase in the aging population, osteoarthritis (OA), especially knee osteoarthritis (KOA), has increasingly become one of the diseases affecting the quality of life of the elderly. As the pathogenesis of KOA is still unclear, Western medicine treatment lacks specificity, and surgical treatment is difficult to cover all patients. Therefore, in recent years, traditional Chinese medicine (TCM) for the conservative treatment of KOA has received increasing attention. The advantages of TCM are clear, such as relief of symptoms, fewer adverse reactions, and wider applicability to patients. This paper mainly discusses the research progress in single-herb TCM and TCM compounds for KOA, aiming to demonstrate the effectiveness of TCM in the treatment of KOA. It also provides ideas for future research on the treatment of KOA by TCM and the pathogenesis of knee osteoarthritis.