Object motion tracking with remote device

Patent metadata

Field	Value
Publication number	US-11513605-B2
Application number	US-202016949268-A
Country	US
Kind code	B2
Filing date	Oct 22, 2020
Priority date	Feb 1, 2016
Publication date	Nov 29, 2022
Grant date	Nov 29, 2022

Abstract

Examples of mixed reality computing devices that utilize remote sensors and local sensors as input devices are disclosed. In one example, a mixed reality computing device comprises an image sensor, a remote input device, a processor, and storage comprising stored instructions. The stored instructions are executable by the processor to perform object motion tracking and environmental tracking based on output from the image sensor, and in response to detecting that the remote input device is in use, adjust a parameter of the motion tracking while maintaining the environmental tracking.

First claim

The invention claimed is:

1. A mixed reality computing system, comprising:
a head-mounted display device;
a remote input device;
a plurality of sensors distributed between the head-mounted display device and the remote input device, the plurality of sensors comprising an image sensor;
a processor; and
storage comprising stored instructions executable by the processor to
in response to determining that the remote input device is in use, activate one or more gesture motion filters and perform object motion tracking based on output from one or more sensors of the plurality of sensors and the one or more gesture motion filters, wherein the one or more gesture motion filters comprise one or more of a first gesture motion filter to filter out detected object motion occurring within a threshold distance of the image sensor and a second gesture motion filter to filter out detected object motion that stays within a field of view of the image sensor for less than a threshold amount of time; and
in response to determining that the remote input device is not in use, deactivate the one or more gesture motion filters.

2. The mixed reality computing system of claim 1 , wherein the instructions are executable to, in response to determining the remote input device is in use, reduce a frame rate of the object motion tracking, and in response to determining that the remote input device is not in use, increase the frame rate of the object motion tracking.

3. The mixed reality computing system of claim 1 , wherein the remote input device comprises an inertial measurement unit (IMU) and a user-actuatable input device, and wherein the instructions are further executable to detect that the remote input device is in use based on one or more of detecting that the remote input device is in the field of view of the image sensor, receiving a user input via the user-actuatable input device of the remote input device, and receiving output from the IMU of the remote input device.

4. The mixed reality computing system of claim 1 , wherein the instructions are further executable to perform an action based on the object motion tracking.

5. A method for adjusting gesture motion filters in a mixed reality computer system comprising a head-mounted display device, a remote input device, and a plurality of sensors distributed between the head-mounted display device and the remote input device, the method comprising:
determining that the remote input device is in use;
reducing a frame rate of object motion tracking when the remote input device is in use;
activating one or more gesture motion filters;
performing the object motion tracking based on output from one or more sensors of the plurality of sensors and the one or more gesture motion filters;
determining that the remote input device is not in use;
increasing the frame rate of the object motion tracking when the remote input device is not in use; and
deactivating the one or more gesture motion filters.

6. The method of claim 5 , further comprising one or more of
filtering out detected object motion occurring within a threshold distance of an image sensor, or
filtering out detected object motion that stays within a field of view of the image sensor for less than a threshold amount of time.

7. The method of claim 5 , wherein determining that the remote input device is in use further comprising detecting that the remote input device is in a field of view of an image sensor.

8. The method of claim 5 , wherein determining that the remote input device is in use further comprising receiving a user input via a user-actuatable input device of the remote input device.

9. The method of claim 5 , wherein determining that the remote input device is in use further comprising receiving an output from an inertial measurement unit (IMU) of the remote input device.

10. The method of claim 5 , further comprising executing an action based on the object motion tracking.

11. A mixed reality computing system, comprising:
a head-mounted display device;
a remote input device;
a plurality of sensors distributed between the head-mounted display device and the remote input device;
a processor; and
storage comprising stored instructions executable by the processor to
in response to determining that the remote input device is in use, activate one or more gesture motion filters, perform object motion tracking based on output from one or more sensors of the plurality of sensors and the one or more gesture motion filters, and reduce a frame rate of the object motion tracking; and
in response to determining that the remote input device is not in use, deactivate the one or more gesture motion filters, and increase the frame rate of the object motion tracking.

12. The mixed reality computing system of claim 11 , wherein the one or more gesture motion filters comprise one or more of a first gesture motion filter to filter out detected object motion occurring within a threshold distance of an image sensor and a second gesture motion filter to filter out detected object motion that stays within a field of view of the image sensor for less than a threshold amount of time.

13. The mixed reality computing system of claim 11 , wherein the remote input device comprises an inertial measurement unit (IMU) and a user-actuatable input device, and wherein the instructions are further executable to detect that the remote input device is in use based on one or more of detecting that the remote input device is in a field of view of an image sensor, receiving a user input via the user-actuatable input device of the remote input device, and receiving output from the IMU of the remote input device.

14. The mixed reality computing system of claim 11 , wherein the instructions are further executable to perform an action based on the object motion tracking.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Potts Lori Ann
• Cherkashin Lev
• Rohn David
• Velat Steven James
• Goris Andrew C
• Fullam Scott Francis
• Legg Travis Scott
• Haskins Craig
• Tichenor James

Classifications

• G06F3/0383

  Signal control means within the pointing device · CPC title

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G02B2027/0138

  comprising image capture systems, e.g. camera · CPC title

• G02B2027/0187

  slaved to motion of at least a part of the body of the user, e.g. head, eye · CPC title