Unlike deep learning which requires large training datasets, correlation filter-based trackers like Kernelized Correlation Filter (KCF) use implicit properties of tracked images (circulant structure) for training in real time. Despite their popularity in tracking applications, there exists significant drawbacks of the tracker in cases like occlusions and out-of-view scenarios. This paper attempts to address some of these drawbacks with a novel RGB-D Kernel Correlation tracker in target re-detection. Our target re-detection framework not only re-detects the target in challenging scenarios but also intelligently adapts to avoid any boundary issues. Our results are experimentally evaluated using (a) standard dataset and (b) real time using the Microsoft Kinect V2 sensor. We believe this work will set the basis for improvement in the effectiveness of kernel-based correlation filter trackers and will further the development of a more robust tracker.