Tracking of position and orientation of objects in virtual reality systems

Patent metadata

Field	Value
Publication number	US-10444865-B2
Application number	US-201815964499-A
Country	US
Kind code	B2
Filing date	Apr 27, 2018
Priority date	May 1, 2017
Publication date	Oct 15, 2019
Grant date	Oct 15, 2019

Abstract

Techniques of tracking a hand controller in a VR system involves detecting, by photodiodes in the hand controller, patterns of diffuse radiation generated by diffuse LEDs in the HMD. Such techniques can also include comparing the detected patterns to those simulated offline previously and represented in a lookup table (LUT). By looking up the detected patterns in the LUT, the VR system may determine the position and/or orientation of the hand controller with sub-millimeter accuracy. Some advantages of the improved techniques can be in the simplicity and/or low cost of the components without sacrificing accuracy in deriving the position and orientation of the hand controller.

First claim

What is claimed is:

1. A method, comprising:
receiving, by processing circuitry of a virtual reality (VR) system including a hand controller and a head-mounted display (HMD), a pulse of diffuse electromagnetic radiation at a plurality of detectors included in the hand controller from a plurality of radiation transmitters included in the HMD, each of the plurality of detectors receiving the pulse of diffuse electromagnetic radiation from a different perspective than a perspective from which another detector of the plurality of detectors receives the pulse of diffuse electromagnetic radiation, the perspective from which each of the plurality of detectors receives the pulse of diffuse electromagnetic radiation being based on a position of that detector in the hand controller;
generating, by the processing circuitry, a plurality of digital signals, each of the plurality of digital signals being generated at a respective detector of the plurality of detectors based on the perspective from which the diffuse electromagnetic radiation is received at that detector; and
obtaining, by the processing circuitry, a position and an orientation of the hand controller based on the plurality of digital signals.

2. The method as in claim 1 , wherein obtaining the position and orientation of the hand controller based on the plurality of digital signals includes:
comparing a digital signal of the plurality of digital signals to a plurality of signal representations of a lookup table (LUT), each of the plurality of signal representations corresponding to a respective position and orientation of the hand controller.

3. The method as in claim 2 , wherein each predetermined signal representation of the plurality of signal representations of the LUT includes a respective signal portion for each of the plurality of radiation transmitters placed in the HMD.

4. The method as in claim 2 , wherein each predetermined signal representation of the plurality of signal representations of the LUT corresponds to a time at which the pulse of diffuse radiation is transmitted from the plurality of radiation transmitters in the HMD.

5. The method as in claim 4 , wherein each of the plurality of digital signals includes a first portion and a second portion;
wherein comparing the digital signal to the predetermined signal representation of the plurality of signal representations of the LUT includes:
producing the time at which the pulse of diffuse radiation is transmitted from the plurality of radiation transmitters in the HMD based on the first portion of the digital signal; and
producing the position and orientation of the hand controller based on the second portion of the signal and the time at which the pulse of radiation is transmitted from the plurality of radiation transmitters in the HMD.

6. The method as in claim 2 , wherein the plurality of radiation transmitters are arranged in a specified angular pattern within the HMD.

7. The method as in claim 2 , wherein the processing circuitry includes a photodiode,
wherein the method further comprises generating the LUT offline by performing a simulation of a response of the photodiode to illumination from each of the radiation transmitters in the HMD, the photodiode response having a transfer function that has a noise profile and a signal profile, the signal profile being zero outside of a specified frequency band having an upper frequency and a lower frequency, the noise profile including 1/f noise below a critical frequency below the lower frequency of the frequency band of the signal profile.

8. The method as in claim 2 , wherein comparing the digital signal to the plurality of signal representations of the LUT includes:
generating a heat map indicative of the closeness of a signal representation in the LUT to the digital signal, the heat map including a hot region in which signal representations correspond to the digital signal and a cool region in which signal representations do not correspond to the digital signal; and
selecting a signal representation in the hot region of the heat map.

9. The method as in claim 8 , wherein the heat map is plotted in three-dimensional position space, and
wherein selecting the signal representation in the hot region of the LUT includes identifying a position of the hand controller as a coordinate triplet in the hot region of the heat map.

10. The method as in claim 9 , wherein selecting the signal representation in the hot region of the LUT further includes:
generating a second heat map at the coordinate triplet in the hot region of the heat map identified as the position of the hand controller, the second heat map being plotted in a three-dimensional pitch-yaw-roll angle space, the second heat map including a hot region in which signal representations correspond to the digital signal and a cool region in which signal representations do not correspond to the digital signal; and
selecting a signal representation in the hot region of the second heat map.

11. The method as in claim 2 , wherein the hand controller includes (i) a transimpedance amplifier (TIA) configured to produce a first current from the pulse of diffuse electromagnetic radiation and a second current from external radiation received from radiation sources external to the plurality of radiation transmitters placed in the HMD and (ii) an error amplifier configured to cancel the second current;
wherein the method further comprises:
receiving the second current at the error amplifier, the second current producing a second voltage at the error amplifier;
in response to receiving the second current, generating a control voltage proportional to a difference between the second voltage and a reference voltage; and
generating a negative current from the control voltage, the negative current drawing away the second current from the TIA.

12. A computer program product comprising a nontransitory storage medium, the computer program product including code that, when executed by processing circuitry of a hand controller of a virtual reality (VR) system that also includes a head-mounted display (HMD), causes the processing circuitry to perform a method, the method comprising:
receiving a pulse of diffuse electromagnetic radiation at a plurality of detectors included in the hand controller from a plurality of radiation transmitters included in the HMD, each of the plurality of detectors receiving the pulse of diffuse electromagnetic radiation from a different perspective than a perspective from which another detector of the plurality of detectors receives the pulse of diffuse electromagnetic radiation, the perspective from which each of the plurality of detectors receives the pulse of diffuse electromagnetic radiation being based on a position of that detector in the hand controller;
generating a plurality of digital signals, each of the plurality of digital signals being generated at a respective detector of the plurality of detectors from based on the perspective from which the diffuse electromagnetic radiation is received at that detector: and obtaining a position and an orientation of the hand controller based on the plurality of digital signals.

13. The computer program product as in claim 12 , wherein obtaining the position and orientation of the hand controller based on the plurality of digital signals includes:
comparing a digital signal of the plurality of digital signals to a plurality of signal representations of a lookup table (LUT), each of the plurality of signal representations corresponding to a respective position and orientation of the hand controller.

14. The computer program product as in claim 13 , whe

Assignees

• Google Llc

Inventors

• Kirmani Ghulam
• Colaco Andrea
• Villaran Luis
• Trehan Chintan

Classifications

• G02B2027/0132

  comprising binocular systems · CPC title

• G02B2027/0138

  comprising image capture systems, e.g. camera · CPC title

• G02B27/0172

  characterised by optical features · CPC title

• G08C2201/32

  Remote control based on movements, attitude of remote control device · CPC title

• G06F3/0325

  using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image · CPC title