The fully convolution network is a very powerful visual model that can be used to extract features in an image. We improved a network model that can be used for end-to-end, pixel-to-pixel training to extract target motion trajectories in infrared images. The dataset used in our training comes from the simulation dataset produced by the public infrared dataset combined with the simulation trajectory. In order to enhance the model’s robustness, we add the pepper and salt noise and white noise to the simulated image, and use image augmentation to increase the number of the image. We achieved highly train and test accuracy in our simulation dataset.