Remote photoplethysmography (rPPG) is a video-based heart rate measurement technology, which is widely used in special scenes where contact equipment is difficult to measure. However, the rPPG signal is very weak, and it is easily affected by factors such as uneven environmental illumination changes and the tester’s head movement, which leads to the poor robustness of the existing methods in natural scenes. A deep learning model based on vision transformer is proposed to segment the facial skin region to generate the spatiotemporal feature map of the video sequence, inputs the feature map into the model for rPPG physiological feature extraction, and then fits the rPPG signal. The experiments verify the effectiveness of the method on mixed data sets and can ensure that the model has a high degree of signal fitting while significantly reducing the computational complexity of the transformer.