Distinguishing the target from the background, judging target occlusion, and real-time processing are the problems that the visual tracking algorithm still needs to solve. Color information and position information of the target block are fused as new features to track the target under the framework of particle filtering. First, the hues, saturation, value space, and color integral graph of the image are constructed. The vector representation of the target is obtained on the color integral image by sparse matrix. Then, candidate particles are produced by a particle filter and the sampling mode of particles is adjusted by a uniform acceleration model. The difference of particles reflects the position and scale change of the target. Finally, the candidate with the smallest eigenvector projection error is taken as the tracking target and the feature template is updated based on the tracking results. The presented algorithm can be used to track a single target in the color image sequence and has some robustness to the scale change, occlusion, and morphological change of the target. Experiment results on public datasets show that the proposed algorithm performs favorably in both speed and tracking effect when compared with other conventional trackers.