In this paper, we propose an improved Color-LBP(local binary pattern) joint histogram model for real-time object tracking. Our main contribution is twofold. First, we introduce the tri-state transition method to extract a noise robust LBP suitable for object tracking. Second, instead of using only the 5 major uniform LBPs, we also incorporate the minor uniform and the non-uniform patterns to create a robust LBP histogram model. Experiment results show that the proposed algorithm outperforms the conventional algorithm.