Object motion tracking with remote device

Patent metadata

Field	Value
Publication number	US-10908694-B2
Application number	US-201615012694-A
Country	US
Kind code	B2
Filing date	Feb 1, 2016
Priority date	Feb 1, 2016
Publication date	Feb 2, 2021
Grant date	Feb 2, 2021

Abstract

Examples of mixed reality computing devices that utilize remote sensors and local sensors as input devices are disclosed. In one example, a mixed reality computing device comprises an image sensor, a remote input device, a processor, and storage comprising stored instructions. The stored instructions are executable by the processor to perform object motion tracking and environmental tracking based on output from the image sensor, and in response to detecting that the remote input device is in use, adjust a parameter of the motion tracking while maintaining the environmental tracking.

First claim

The invention claimed is:

1. A mixed reality computing device, comprising:
an image sensor;
a processor; and
storage comprising stored instructions executable by the processor to
when a remote input device is not being used to provide gesture input, perform object motion tracking at a higher image frame rate and environmental tracking at a lower image frame rate based on output from the image sensor, the remote input device comprising an inertial measurement unit (IMU) and a user-actuatable input device;
detect use of the remote input device to provide gesture input; and
in response to detecting use of the remote input device to provide gesture input, adjust an image frame rate used to perform the object motion tracking such that the object motion tracking is performed at an adjusted image frame rate by deactivating the object motion tracking based on output from the image sensor while maintaining the lower image frame rate used for the environmental tracking, and perform the object motion tracking based on output from the IMU of the remote input device.

2. The mixed reality computing device of claim 1 , wherein the instructions are further executable to adjust the image frame rate used to perform the object motion tracking in response to a change in a selected sensor operation mode.

3. The mixed reality computing device of claim 1 , wherein the instructions are executable to, further in response to detecting use of the remote input device to provide gesture input, activate one or more gesture filters to filter out motion detected in front of the image sensor.

4. The mixed reality computing device of claim 1 , further comprising a light source, and wherein the instructions are executable to perform the object motion tracking by activating the light source at a first brightness, and to perform the environmental tracking by activating the light source at a second brightness.

5. The mixed reality computing device of claim 1 , wherein the instructions are further executable to adjust the image frame rate used to perform the object motion tracking based at least on power consumption of the image sensor.

6. The mixed reality computing device of claim 1 , wherein the instructions are further executable to detect use of the remote input device based on one or more of detecting that the remote input device is in a field of view of the image sensor, receiving a user input via the user-actuatable input device of the remote input device, and receiving output from the IMU of the remote input device.

7. The mixed reality computing device of claim 1 , wherein the instructions are further executable to perform an action on the mixed reality computing device based on the environmental tracking and the object motion tracking.

8. The mixed reality computing device of claim 1 , wherein the remote input device further comprises one or more image sensors.

9. On a mixed reality computing device comprising an image sensor, a method comprising:
when a remote input device is not being used to provide gesture input, performing object motion tracking at a higher image frame rate and environmental tracking at a lower image frame rate based on output from the image sensor, the remote input device comprising an inertial measurement unit (IMU) and a user-actuatable input device;
detecting use of the remote input device to provide gesture input; and
in response to detecting use of the remote input device to provide gesture input, adjusting an image frame rate used to perform the object motion tracking such that the object motion tracking is performed at an adjusted image frame rate by deactivating the object motion tracking based on output from the image sensor, while maintaining the lower image frame rate used for the environmental tracking, and performing the object motion tracking based on output from the IMU of the remote input device.

10. The method of claim 9 , further comprising adjusting the image frame rate used to perform the object motion tracking in response to a change in a selected sensor operation mode.

11. The method of claim 9 , further comprising, in response to detecting use of the remote input device to provide gesture input, activating one or more gesture filters to filter out motion detected in front of the image sensor.

12. The method of claim 9 , wherein the mixed reality computing device comprises a light source, and further comprising performing the object motion tracking by activating the light source at a first brightness, and performing the environmental tracking by activating the light source at a second brightness.

13. The method of claim 9 , further comprising adjusting the image frame rate used to perform the object motion tracking based at least on power consumption of the image sensor.

14. The method of claim 9 , wherein detecting use of the remote input device comprises one or more of detecting that the remote input device is in a field of view of the image sensor, receiving a user input via the user-actuatable input device of the remote input device, and receiving output from the IMU of the remote input device.

15. The method of claim 9 , further comprising performing an action on the mixed reality computing device based on the environmental tracking and the object motion tracking.

16. A computer-readable storage device comprising instructions executable by a mixed reality computing device comprising an image sensor to:
when a remote input device is not being used to provide gesture input, perform object motion tracking at a higher image frame rate and environmental tracking at a lower image frame rate based on output from the image sensor, the remote input device comprising an inertial measurement unit (IMU) and a user-actuatable input device;
detect use of the remote input device to provide gesture input; and
in response to detecting use of the remote input device to provide gesture input, adjust an image frame rate used to perform the object motion tracking such that the object motion tracking is performed at an adjusted image frame rate by deactivating the object motion tracking based on output from the image sensor while maintaining the lower image frame rate used for the environmental tracking, and perform the object motion tracking based on output from the IMU of the remote input device.

17. The computer-readable storage device of claim 16 , wherein the instructions are further executable to adjust the image frame rate used to perform the object motion tracking in response to a change in a selected sensor operation mode.

18. The computer-readable storage device of claim 16 , wherein the instructions executable to, further in response to detecting use of the remote input device to provide gesture input, activate one or more gesture filters to filter out motion detected in front of the image sensor.

19. The computer-readable storage device of claim 16 , wherein the instructions are further executable to perform the object motion tracking by activating a light source of the mixed reality computing device at a first brightness, and perform the environmental tracking by activating the light source at a second brightness.

20. The computer-readable storage device of claim 16 , wherein the instructions are further executable to adjust the image frame rate used to perform the object motion tracking based at least on power consumption of the image sensor.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Potts Lori Ann
• Cherkashin Lev
• Rohn David
• Velat Steven James
• Goris Andrew C
• Fullam Scott Francis
• Legg Travis Scott
• Haskins Craig
• Tichenor James

Classifications

• G06F2203/0384

  Wireless input, i.e. hardware and software details of wireless interface arrangements for pointing devices · CPC title

• G06F3/0383

  Signal control means within the pointing device · CPC title

• G06F3/0346

  with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors · CPC title

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title