In this paper we present a novel visual tracking algorithm, in which object tracking is achieved by using subspace learning and Huber loss regularization in a particle filter framework. The changing appearance of tracked target is modeled by Principle Component Analysis (PCA) basis vectors and row group sparsity. This method takes advantage of the strengths of sub-space representation and explicitly takes the underlying relationship between particle candidates into consideration in the tracker. The representation of each particle is learned via the multi-task sparse learning method. Huber loss function is employed to model the error between candidates and templates, yielding robust tracking. We utilize the Alternating Direction Method of Multipliers (ADMM) to solve the proposed representation model. In experiments we tested sixty representative video sequences that reflect the specific challenges of tracking and used both qualitative and quantitative metrics to evaluate the performance of our tracker. The experiment results demonstrated that the proposed tracking algorithm achieves superior performance compared to nine state-of-the-art tracking methods.