Meta Platforms, Inc.
Tracking using controller cameras

Abstract:

A virtual reality (VR) system tracks the location and position of a controller using image sensors on a headset and a controller. The headset and controller provide a first and second view of a user's environment. Using its camera, the headset generates a map of the environment and identifies its location within it. Based headset's location, the VR system generates a simulated world of the environment and displays it to the user. The headset also estimates the location of the controller. Based on the estimated location, the headset sends a portion of the map to the controller. The controller determines its pose using the portion of the map, the image sensors, and additional sensors. The controller sends its pose and an updated portion of the map to the headset. Based on the controller's pose and updated portion of the map, the VR system modifies the content displayed to the user.