Apple Inc.
Depth rate up-conversion

Abstract:

Various implementations disclosed herein include devices, systems, and methods that create additional depth frames in the circumstance wherein a depth camera runs at a lower frame rate than a light intensity camera. Rather than upconverting the depth frames by simply repeating a previous depth camera frame, additional depth frames are created by adjusting some of the depth values of a prior frame based on the RGB camera data (e.g., by "dragging" depths from their positions in the prior depth frame to new positions for a new frame). Specifically, a contour image (e.g., identifying interior and exterior outlines of a hand with respect to a virtual cube that the hand occludes) is generated based on a mask (e.g., occlusions masks identifying where the hand occludes the virtual cube). Changes in the contour image are used to determine how to adjust (e.g., drag) the depth values for the additional depth frames.