Meta Platforms, Inc.
Deep multi-scale video prediction

Abstract:

In one embodiment, a method includes receiving a plurality of input frames of a video sequence associated with a time t, predicting, using a convolutional network, one or more future frames of the video sequence from the plurality of input frames, wherein the convolutional network is trained with randomly selected temporal sequences of a n.times.m grid of pixels from the plurality of input frames exhibiting a threshold of optical flow. In addition, the training may comprise randomly selecting temporal sequences of a n.times.m grid of pixels from the plurality of input frames exhibiting a threshold of optical flow.