NVIDIA Corporation
SENSOR SIMULATION AND LEARNING SENSOR MODELS WITH GENERATIVE MACHINE LEARNING METHODS

Abstract:

In various examples, a sensor model may be learned to predict virtual sensor data for a given scene configuration. For example, a sensor model may include a deep neural network that supports generative learning--such as a generative adversarial network (GAN). The sensor model may accept an encoded representation of a scene configuration as an input using any number of data structures and/or channels (e.g., concatenated vectors, matrices, tensors, images, etc.), and may output virtual sensor data. Real-world data and/or virtual data may be collected and used to derive training data, which may be used to train the sensor model to predict virtual sensor data for a given scene configuration. As such, one or more sensor models may be used as virtual sensors in any of a variety of applications, such as in a simulated environment to test features and/or functionality of one or more autonomous or semi-autonomous driving software stacks.