Royal Bank of Canada
SYSTEM AND METHOD FOR UNSUPERVISED SCENE DECOMPOSITION USING SPATIO-TEMPORAL ITERATIVE INFERENCE

Abstract:

Systems and methods for unsupervised multi-object scene decomposition that involve a spatio-temporal amortized inference model for multi-object video decomposition. Systems and methods involve a new spatio-temporal iterative inference framework to jointly model complex multi-object representations and the explicit temporal dependencies between the frames. Those dependencies improve overall quality of decomposition, encode information about object dynamics and can be used to predict future trajectories of each object separately. Additionally, the model can generate precise estimations and output data even without color information. The model has scene decomposition, segmentation and future prediction capabilities. The processor can use the model to simulate future frames of the scene data.