Being characterized by high resolution and high frame rate imaging, Video Synthetic Aperture Radar (ViSAR) can continuously obtain the approximate video information showed interest in targets in the area on the ground, laying a foundation for the rapid development of target recognition and tracking technology based on SAR images. In order to meet the increasingly complex application requirements, the multi-target tracking technology has gradually developed and matured. Aimed at the problems that the state of each moving target in the process of multi-target tracking has space-time variability, and the number of targets has randomness, a multi-target tracking algorithm is established based on the Random Finite Set (RFS). On this basis, the Probability Hypothesis Density (PHD) algorithm under the Bayesian framework is discussed, and the Gaussian Mixture Probability Hypothesis Density (GM-PHD) filtering algorithm is studied and implemented under condition of the Gaussian mixture model, and then the RFS-based multi-target tracking algorithm is further implemented. The results show that the algorithm is valid under condition of the complex environment background.