Universal tracking module

Patent metadata

Field	Value
Publication number	US-12475577-B2
Application number	US-202318351026-A
Country	US
Kind code	B2
Filing date	Jul 12, 2023
Priority date	Jul 12, 2023
Publication date	Nov 18, 2025
Grant date	Nov 18, 2025

Abstract

Aspects of the present disclosure are directed to a universal tracking module (UTM) capable of tracking six degrees of freedom of objects. In artificial reality environments, six degrees of freedom tracking refers to the ability to track an object moving and rotating freely in three-dimensional space along three axes of movement and rotation. Six degrees of freedom tracking provides high precision in tracking movement and rotation in 3D space, enabling more accurate and realistic representations of objects and interactions in virtual and augmented reality environments, thereby allowing for more realistic and immersive experiences. In examples, the UTM includes a plurality of light-emitting diodes such that an image of the UTM can be utilized to determine the position and orientation of the UTM and the object to which the UTM is attached.

First claim

We claim:

1 . A method for tracking a position and orientation of an object, the method comprising:
acquiring an image of a tracking module affixed to an object, the tracking module including one or more illuminated light-emitting diodes captured in the image; identifying one or more locations, with respect to the tracking module, of the one or more illuminated light-emitting diodes; receiving position and orientation data from an inertial measurement unit associated with the tracking module; determining a position and orientation of the tracking module based on the position and orientation data received from the inertial measurement unit and the identified one or more locations of the one or more light-emitting diodes; translating the determined position and orientation of the tracking module to position and orientation information of the object by:
applying a process that maps a) a three-dimensional structure of the tracking module and the determined position and orientation of the tracking module to b) a three-dimensional structure of the object, thereby determining the translated position and orientation information of the object, wherein the process is applied using a defined relationship between a coordinate system of the tracking module and a coordinate system associated with the three-dimensional structure of the object; and
rendering a representation of the object, in an artificial reality environment, based on the translated position and orientation information of the object.

2 . The method of claim 1 , further comprising:
determining a change in position and orientation of the tracking module based on a previous determined position and orientation of the tracking module; applying a mathematical operation to map the determined change in position and orientation of the tracking module to a change in position and orientation of the object; and determining the translated position and orientation information of the object as the change in position and orientation of the object, wherein the representation of the object is rendered using the three-dimensional structure of the object.

3 . The method of claim 1 , further comprising:
providing light-emitting diode configuration information to the tracking module, the light-emitting diode configuration information defining a blink rate for the one or more light-emitting diodes of the tracking module; and acquiring the image of the tracking module with an imaging device associated with a head-mounted display, wherein the blink rate for the one or more light-emitting diodes of the tracking module corresponds to a frame rate of the imaging device, the head-mounted display renders the representation of the object in the artificial reality environment, and the representation comprises a three-dimensional representation of the object.

4 . The method of claim 3 , further comprising:
matching, based on the position and orientation data received from the inertial measurement unit, the identified locations of the one or more illuminated light-emitting diodes to at least one light-emitting diode pattern location from a light-emitting diode pattern for the tracking module, wherein the light-emitting diode pattern for the tracking module specifies an arrangement of light-emitting diodes positioned around the tracking module; and determining the position and orientation of the tracking module using the matched at least one light-emitting diode pattern location.

5 . The method of claim 4 , wherein,
the one or more identified locations of the one or more illuminated light-emitting diodes comprise at least two identified locations of at least two illuminated light-emitting diodes captured in the image, at least two light-emitting diode pattern locations matched to the at least two identified locations, the at least matched two light-emitting diode pattern locations corresponding to a sub-pattern of the light-emitting diode pattern for the tracking module; and the position and orientation of the tracking module is determined using the matched at least two light-emitting diode pattern locations.

6 . The method of claim 1 , further comprising:
processing the image of the tracking module, wherein the processing comprises suppressing one or more sources of light captured in the image before identifying a location of the one or more illuminated light-emitting diodes.

7 . The method of claim 6 , wherein identifying a location, with respect to the tracking module, of the one or more illuminated light-emitting diodes includes:
performing blob segmentation and identification to locate one or more pixels associated with a light-emitting diode; and matching at least one blob identified by performing the blob segmentation and identification with a light-emitting diode of a light-emitting diode pattern that is specific to the tracking module.

8 . A non-transitory computer-readable storage medium storing instructions that, when executed by a computing system, cause the
computing system to perform a process for tracking a position and orientation of an object, the process comprising: receiving position and orientation data associated with a tracking module; determining a position and orientation of the tracking module based on the position and orientation data received; translating the position and orientation of the tracking module to position and orientation information of the object by:
applying a process that maps a) a three-dimensional structure of the tracking module and the determined position and orientation of the tracking module to b) a three-dimensional structure of the object, thereby determining the translated position and orientation information of the object, wherein the process is applied using a defined relationship between a coordinate system of the tracking module and a coordinate system associated with the three-dimensional structure of the object;
rendering, by a head-mounted display, a three-dimensional representation of the object in an artificial reality environment based on the translated position and orientation information of the object; receiving at least one of an action or command passed from the object to the tracking device via a communication pathway; and altering the representation of the object in accordance with the command or action passed from the object to the tracking device.

9 . The non-transitory computer-readable storage medium of claim 8 , wherein the process further comprises:
acquiring an image of the tracking module affixed to the object, the tracking module including one or more illuminated light-emitting diodes captured in the image; and receiving the position and orientation data from an inertial measurement unit of the tracking module, wherein the determined position and orientation of the tracking module is based on the position and orientation data received from the inertial measurement unit and the identified locations of the one or more light-emitting diodes.

10 . The non-transitory computer-readable storage medium of claim 9 , wherein the process further comprises:
providing light-emitting diode configuration information to the tracking module, the light-emitting diode configuration information defining a blink rate for the one or more light-emitting diodes of the tracking module blink; and acquiring the image of the tracking module with an imaging device associated with the head-mounted display, wherein the blink rate for the one or more light-emitting diodes of the tracking module blink corresponds to a frame rate of the imaging device.

11 . The non-transitory computer-readable storage medium of claim 8 , wherein altering the representation of the object in accor

Assignees

• Meta Platforms Tech Llc

Inventors

• Higgins Jason Andrew
• Jentz Lorenz Henric
• Everist Nicholas Friele
• Boulos Khaled

Classifications

• G06F3/016

  Input arrangements with force or tactile feedback as computer generated output to the user · CPC title

• G06T2207/20021

  Dividing image into blocks, subimages or windows · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06F3/0308

  comprising a plurality of distinctive and separately oriented light emitters or reflectors associated to the pointing device, e.g. remote cursor controller with distinct and separately oriented LEDs at the tip whose radiations are captured by a photo-detector associated to the screen · CPC title

• G06F3/0354 Primary

  with detection of two-dimensional [2D] relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks · CPC title