Event driven sensor (eds) tracking of light emitting diode (led) array

Patent metadata

Field	Value
Publication number	US-2022057859-A1
Application number	US-202117519177-A
Country	US
Kind code	A1
Filing date	Nov 4, 2021
Priority date	Jan 13, 2020
Publication date	Feb 24, 2022
Grant date	—

Abstract

An event driven sensor (EDS) is used for simultaneous localization and mapping (SLAM) and in particular is used in conjunction with a constellation of light emitting diodes (LED) to simultaneously localize all LEDs and track EDS pose in space. The EDS may be stationary or moveable and can track moveable LED constellations as rigid bodies. Each individual LED is distinguished at a high rate using minimal computational resources (no image processing). Thus, instead of a camera and image processing, rapidly pulsing LEDs detected by the EDS are used for feature points such that EDS events are related to only one LED at a time.

First claim

What is claimed is:

1 . A method comprising:
energizing plural lamps one at a time; detecting events from energizing the lamps using an even detector sensor (EDS); and using output of the EDS to generate simultaneous localization and mapping (SLAM) information in an augmented reality (AR) environment at least in part by generating, from the SLAM information, plural motion hypotheses; and selecting one of the hypotheses on the basis that the one of the hypotheses has no outliers for use of the one of the hypotheses for an initial relative pose indication.

2 . The method of claim 1 , comprising movably mounting the lamps.

3 . The method of claim 1 , comprising movably mounting the EDS.

4 . The method of claim 1 , comprising movably mounting the EDS and the lamps.

5 . The method of claim 1 , wherein the lamps comprise light emitting diodes (LED).

6 . The method of claim 1 , wherein the EDS is a first EDS and method comprises using a second EDS to detect events generated by the LEDs for determining stereo SLAM.

7 . The method of claim 1 , wherein the EDS is a first EDS, and the method comprises using information only from the first EDS and no other EDS to determine monocular SLAM.

8 . The method of claim 7 , comprising:
determining initial triangulation and pose estimation at least in part by: receiving at least first and second frames from the EDS generated using plural events from different LEDs; determining, from the first and second frames, a fundamental matrix; computing an essential matrix from the fundamental matrix; decomposing the essential matrix into four motion hypotheses; selecting a first one of the four motion hypotheses based on the first motion hypothesis having no outliers; and using the first motion hypothesis execute SLAM by triangulating new LED positions using the first motion hypothesis as a reference.

9 . A system comprising:
at least one light emitting diode (LED) assembly configured to cause individual LEDs in the assembly to emit light one LED at a time; at least one event driven sensor (EDS) configured to output signals representing changes in illumination intensity responsive to LED emissions; and at least one processor configured with instructions to: receive signals from at least one inertial measurement unit (IMU); and generate object position and/or orientation information based at least in part on the signals from the EDS and the signals from the IMU.

10 . The system of claim 9 , wherein the instructions are executable to determine a true orientation with respect to vertical using the signals from the IMU.

11 . The system of claim 9 , wherein the instructions are executable to determine, using the signals from the IMU, angular velocity and use the angular velocity to identify an orientation change.

12 . The system of claim 9 , wherein the EDS is moveable, and the LED assembly is stationary.

13 . The system of claim 9 , wherein the EDS is not moveable, and the LED assembly is moveable.

14 . The system of claim 9 , wherein the EDS and LED assembly are both moveable.

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Bashkirov Sergey

Classifications

• G06F3/0325

  using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/0346

  with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors · CPC title

• G01S5/16

  using electromagnetic waves other than radio waves · CPC title

• G06F3/012 Primary

  Head tracking input arrangements · CPC title