The COVID-19 virus is increasingly crucial to human health since new variants appear frequently. Detection of COVID-19 through respiratory sound has been an important area of research. This study analyzes respiratory sounds using novel accumulated bi-spectral features. The principal domain bispectrum is used for computing accumulated bispectrum. The resulting magnitude bispectrum is used in forming the bispectral image. In this work, a convolutional neural network (CNN) and ResNet-50 algorithms are designed to classify respiratory sounds as either COVID-19 or healthy. The performance of the proposed method is compared with the state-of-the-art methods. The proposed CNN-based method achieves the highest accuracy of 87.68% for shallow breath sounds, and ResNet-50 achieves the highest accuracy of 87.62% for deep breath sounds. Similarly, proposed methods gives the improved performance for other respiratory sounds.