Amazon.com, Inc.
Detailed 3D estimation from a single image

Abstract:

A system configured to perform an accurate and fast estimation of an object shape from a single input image. The system may process image data representing a first surface of an object using image-to-image translation techniques. A first trained model may generate depth information for the object, such as front distance estimates and back distance estimates. The system may use the depth information to generate an output mesh shaped like the object, such as, in the case of a pliable object a reposable avatar. The system may improve depth estimation by including a loss on surface normals in the first trained model. A second trained model may generate color information to be applied to the output mesh to accurately represent the object. The output mesh may include detailed geometry and appearance of the object, useful for a variety of purposes such as gaming, virtual/augmented reality, virtual shopping, and other implementations.