We introduce a novel kernel low-rank algorithm to recover free-breathing and ungated dynamic MRI data from highly undersampled measurements. The image frames in the free breathing and ungated dataset are assumed to be points on a bandlimited manifold. We show that the non-linear features of these images satisfy annihilation conditions, which implies that the kernel matrix derived from the dataset is low-rank. We penalize the nuclear norm of the feature matrix to recover the images from highly undersampled measurements. The regularized optimization problem is solved using an iterative reweighted least squares (IRLS) algorithm, which alternates between the update of the Laplacian matrix of the manifold and the recovery of the signals from the noisy measurements. To improve computational efficiency, we use a two step algorithm using navigator measurements. Specifically, the Laplacian matrix is estimated from the navigators using the IRLS scheme, followed by the recovery of the images using a quadratic optimization. We show the relation of this two step algorithm with our recent SToRM approach, thus reconciling SToRM and manifold regularization methods with algorithms that rely on explicit lifting of data to a high dimensional space. The IRLS based estimation of the Laplacian matrix is a systematic and noise-robust alternative to current heuristic strategies based on exponential maps. We also approximate the Laplacian matrix using a few eigen vectors, which results in a fast and memory efficient algorithm. The proposed scheme is demonstrated on several patients with different breathing patterns and cardiac rates.