Pattern projection-based three-dimensional (3D) measurement systems are widely used for contactless, nondestructive, and full-field optical 3D shape measurements. 3D reconstruction is performed between one camera and the projector or between two cameras by detection and triangulation of corresponding image points. In order to record fast processes, such as people in motion or even explosions and crashes, we have recently developed a 3D stereo sensor consisting of two high-speed cameras and a GOBO projection-based high-speed pattern projector. The system, which works in the visible wavelength range (VIS), enables us to successfully reconstruct 3D point clouds of athletes in action, crash tests, or airbag inflations. However, as such processes usually exhibit local temperature changes, simultaneously measuring the surface temperature would be of great benefit. Therefore, we have extended our existing high-speed 3D sensor by including an additional high-speed longwave infrared (LWIR) camera, which detects radiation in the spectral range between 7.5 and 12 μm. The setup allows us to map the recorded temperature data onto the reconstructed 3D points. We present the design of this novel 5D (three spatial coordinates, temperature, and time) sensor and the process of simultaneously calibrating the VIS cameras and the LWIR camera in a common coordinate system. Moreover, we show first promising measurements of an inflating airbag and a basketball player conducted at a frame rate of 1 kHz.