Logitech International S.A.
Method and system for object segmentation in a mixed reality environment

Abstract:

Methods and systems for providing a mixed reality (MR) interaction are provided. In one example, the method comprises: capturing a first image of a user's appendage and a physical peripheral input device, wherein the user's appendage covers at least a part of the physical peripheral input device; performing a segmentation of the first image into a first region of pixels corresponding to the user's appendage, wherein a second region of pixels corresponds to the physical peripheral input device; generating a composite image from the first image based on rendering, in place of the second region of pixels, pixels representing a virtual peripheral input device, such that the virtual peripheral input device appears in the composite image as at a same spatial location as the physical peripheral input device and covered at least partially by the user's appendage; and displaying the composite image to the user.