Intel Corporation
Methods and apparatus for unsupervised multimodal anomaly detection for autonomous vehicles

Abstract:

An example includes obtaining first sensor data from a first sensor and second sensor data from a second sensor, the first sensor of a first sensor type different than a second sensor type of the second sensor; generating first encoded sensor data based on the first sensor data and second encoded sensor data based on the second sensor data; generating a contextual fused sensor data representation of the first and second sensor data based on the first and second encoded sensor data; generating first and second reconstructed sensor data based on the contextual fused sensor data representation; determining a deviation estimation based on the first and second reconstructed sensor data, the deviation estimation representative of a deviation between: (a) the first reconstructed sensor data, and (b) the first sensor data; and detecting an anomaly in the deviation estimation, the anomaly indicative of an error associated with the first sensor.