Microsoft Corporation
Detecting pose using floating keypoint(s)

Abstract:

In various examples there is an apparatus for detecting position and orientation of an object. The apparatus comprises a memory storing at least one frame of captured sensor data depicting the object. The apparatus also comprises a trained machine learning system configured to receive the frame of the sensor data and to compute a plurality of two dimensional positions in the frame. Each predicted two dimensional position is a position of sensor data in the frame depicting a keypoint, where a keypoint is a pre-specified 3D position relative to the object. At least one of the keypoints is a floating keypoint depicting a pre-specified position relative to the object, lying inside or outside the object's surface. The apparatus comprises a pose detector which computes the three dimensional position and orientation of the object using the predicted two dimensional positions and outputs the computed three dimensional position and orientation.