Machine learning algorithms require datasets that are both massive and varied to train and generalize effectively. However, preparing real-world semantically labeled datasets is a very time-consuming and cumbersome task. Also, training with low volume datasets can lead to compromised performance and poor generalization of such algorithms. This algorithm performance and generalization gap due to limited quantities of real-world data could be decreased with the help of synthetic datasets that are generated with the consideration of real-world features. In this work, a combination of synthetic and real-world datasets is used to demonstrate and assess the performance of simulated-to-real-world transfer learning algorithms where the training is done in synthetic and testing in real-world datasets. The performance is further evaluated with a mixture of real and synthetic datasets. Two simulators are used in this work to generate synthetic images. The first was the Mississippi State University Autonomous Vehicle Simulator (MAVS), a high-fidelity physics-based simulator for Autonomous Ground Vehicle (AGV) in off-road terrain. The MAVS has been used to study machine learning in a variety of applications using both camera and lidar data. In addition to MAVS, the Unreal Engine version 4 (UE4) was used to generate images. Finally, images with a variety of synthetic scene fidelities and real-world images were considered for training the neural network to evaluate the effectiveness of low-fidelity synthetic data and the network performed very well with excellent confidence scores for object detection.
Simulation has become an important enabler in the development and testing of autonomous ground vehicles (AGV), with simulation being used both to generate training data for AI/ML-based segmentation and classification algorithms and to enable in-the-loop testing of the AGV systems that use those algorithms. Furthermore, digital twins of physical test areas provide a safe, repeatable way to conduct critical safety and performance testing of these AI/ML algorithms and their performance on AGV systems. For both these digital twins and the sensor models that use them to generate synthetic data, it is important to understand the relationship between the fidelity of the scene/model and the accuracy of the resulting synthetic sensor data. This work presents a quantitative evaluation of the relationship between digital scene fidelity, sensor model fidelity, and the quality of the resulting synthetic sensor data, with a focus on camera data typically used on AGV to enable autonomous navigation.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.