Meta Platforms, Inc.
Image object extraction and in-painting hidden surfaces for modified viewpoint rendering

Abstract:

In one embodiment, a method includes generating depth map for a reference image and generating a three-dimensional (3D) model for a plurality of objects in the reference image based on the depth map. The method additionally includes determining, out of the objects in the 3D model, a background object having a boundary adjacent to a foreground object. The method also includes determining that at least a portion of a surface of the background object is hidden by the foreground object and extending, in the 3D model, the surface of the background object to include the portion hidden by the foreground object. The method further includes in-paint pixels of the extended surface of the background object with pixels that approximate the portion of the surface of the background object hidden by the foreground object.