Three dimensional object-localization and tracking using ultrasonic pulses with synchronized inertial position determination

Patent metadata

Field	Value
Publication number	US-11269047-B2
Application number	US-201816212503-A
Country	US
Kind code	B2
Filing date	Dec 6, 2018
Priority date	Dec 6, 2017
Publication date	Mar 8, 2022
Grant date	Mar 8, 2022

Abstract

A tracking method is disclosed. The method may include displaying visual content on a screen. A base station may be stationary with respect to the screen while the visual content is being displayed. In contrast, one or more objects may move with respect to the screen while the visual content is being displayed. The one or more objects may be tracked so that the movement thereof may be used to alter the visual content. Such tracking may involve the base station and the one or more objects sending and/or receiving one or more ultrasonic pulses. The tracked object also determines information using an inertial sensor assembly that receives a synchronization signal coordinated with the one or more ultrasonic pulses. Time-difference-of-arrival and/or time-of-flight of the one or more ultrasonic pulses may then be used to estimate a relative location and/or a relative orientation of the one or more objects with respect to the base station in three dimensional space, so that the estimate is fused with information determined by the inertial sensor assembly using the synchronization signal.

First claim

What is claimed is:

1. A method comprising:
moving at least one tracked object with respect to a base station, the at least one tracked object comprising at least one ultrasonic transducer and at least one inertial sensor assembly comprising a three axis accelerometer and a three axis gyroscope, the base station comprising first, second, and third ultrasonic transducers; and
tracking, during the moving, relative distance in three dimensional space between the base station and the at least one tracked object, the tracking comprising
transmitting, by the first ultrasonic transducer, a first ultrasonic pulse,
transmitting, by the second ultrasonic transducer, a second ultrasonic pulse, wherein a time delay between the first ultrasonic pulse and the second ultrasonic pulse is greater than a pulse duration of the first ultrasonic pulse plus a distance between the first and second ultrasonic transducers divided by a speed of sound in an environment surrounding the base station,
transmitting, by the third ultrasonic transducer, a third ultrasonic pulse, wherein a time delay between the second ultrasonic pulse and the third ultrasonic pulse is greater than a pulse duration of the second ultrasonic pulse plus a distance between the second and third ultrasonic transducers divided by the speed of sound,
receiving, by the at least one ultrasonic transducer, the first, second, and third ultrasonic pulses,
using at least one of time-difference-of-arrival and time-of-flight of the first, second, and third ultrasonic pulses to estimate at least one of a relative location and a relative orientation of the at least one tracked object with respect to the base station in three dimensional space,
transmitting, by the base station, a synchronization signal coordinated with at least one of the first ultrasonic pulse, the second ultrasonic pulse, and the third ultrasonic pulse,
receiving, by the at least one tracked object, the synchronization signal
receiving, by the at least one tracked object, the synchronization signal, and
fusing the estimate with information derived from the inertial sensor assembly based at least in part on the synchronization signal.

2. The method of claim 1 , wherein the synchronization signal is received by the inertial sensor assembly.

3. The method of claim 1 , further comprising generating an IMU synchronization signal upon receiving the synchronization signal by the at least one tracked object and providing the IMU synchronization signal to the inertial sensor assembly.

4. The method of claim 3 , wherein the inertial sensor assembly determines a time difference between the received IMU synchronization signal and at least one sensor data sample.

5. The method of claim 4 , wherein the inertial sensor assembly is configured to output the sensor data and the determined time difference.

6. The method of claim 3 , wherein the inertial sensor assembly determines a timestamp of the received IMU synchronization signal based on an internal inertial sensor assembly clock.

7. The method of claim 6 , wherein the inertial sensor assembly is configured to output timestamped sensor data, wherein the timestamp is based on an internal clock, further comprising coordinating the timestamped sensor data with the IMU synchronization signal timestamp.

8. The method of claim 3 , wherein the IMU synchronization signal is generated a predetermined delay after receiving the synchronization signal.

9. The method of claim 3 , wherein the IMU synchronization signal is generated a predetermined delay after receiving the synchronization signal.

10. The method of claim 1 , wherein the fusing comprises a comparison of a timing of the synchronization signal based on a clock of the at least one tracked object and a timing of the synchronization signal based on a clock of the inertial sensor assembly.

11. The method of claim 1 , further comprising generating an ultrasonic trigger signal to start an ultrasonic receive phase of the at least one ultrasonic transducer upon receiving the synchronization signal by the at least one tracked object.

12. The method of claim 11 , further comprising providing the ultrasonic trigger signal to the inertial sensor assembly.

13. The method of claim 1 , wherein the synchronization signal comprises a radio pulse.

14. The method of claim 1 , wherein the synchronization signal comprises an ultrasonic pulse.

15. A system comprising:
a base station comprising first, second, and third ultrasonic transducers;
at least one tracked object that is movable with respect to the base station, the at least one tracked object comprising at least one ultrasonic transducer and an inertial sensor assembly having a three axis accelerometer and a three axis gyroscope; and
at least one processing device programmed to:
invoke transmitting, by the first ultrasonic transducer, a first ultrasonic pulse;
invoke transmitting, by the second ultrasonic transducer, a second ultrasonic pulse, wherein a time delay between the first ultrasonic pulse and the second ultrasonic pulse is greater than a pulse duration of the first ultrasonic pulse plus a distance between the first and second ultrasonic transducers divided by a speed of sound in an environment surrounding the base station;
invoke transmitting, by the third ultrasonic transducer, a third ultrasonic pulse, wherein a time delay between the second ultrasonic pulse and the third ultrasonic pulse is greater than a pulse duration of the second ultrasonic pulse plus a distance between the second and third ultrasonic transducers divided by the speed of sound;
detect receiving, by the at least one ultrasonic transducer, the first, second, and third ultrasonic pulses;
use at least one of time-difference-of-arrival and time-of-flight of the first, second, and third ultrasonic pulses to estimate at least one of a relative location and a relative orientation of the at least one tracked object with respect to the base station in three dimensional space; and
fuse the estimate with information derived from the inertial sensor assembly based at least in part on a synchronization signal transmitted by the base station and received by the at least one tracked object that is coordinated with at least one of the first ultrasonic pulse, the second ultrasonic pulse, and the third ultrasonic pulse.

16. The system of claim 15 , wherein the at least one tracked object is configured to receive the synchronization signal with the inertial sensor assembly.

17. The system of claim 15 , wherein the at least one tracked object is configured to generate an IMU synchronization signal upon receiving the synchronization signal and provide the IMU synchronization signal to the inertial sensor assembly.

18. The system of claim 15 , wherein the at least one processing device is programmed to fuse the estimate with information derived from the inertial sensor assembly by comparing a timing of the synchronization signal based on a clock of the at least one tracked object and a timing of the synchronization signal based on a clock of the inertial sensor assembly.

19. The system of claim 15 , wherein the at least one tracked object is configured to generate an ultrasonic trigger signal to start an ultrasonic receive phase of the at least one ultrasonic transducer upon receiving the synchronization signal by the at least one tracked object and provide the ultrasonic trigger signal to the inertial sensor assembly.

20. The system of claim 15 , wherein the fusion is performed by one or more processors forming part of

Assignees

• Invensense Inc

Inventors

• Riccardi Sebastian
• Przbyla Richard J
• Horsley David A
• Kline Mitchell H

Classifications

• G01C21/1652

  with ranging devices, e.g. LIDAR or RADAR · CPC title

• G01S5/26

  Position of receiver fixed by co-ordinating a plurality of position lines defined by path-difference measurements (G01S5/28 takes precedence) · CPC title

• G01S5/186

  Determination of attitude (using inertial means G01C9/00; control of attitude G05D1/49) · CPC title

• G06F3/033

  Pointing devices displaced or positioned by the user {, e.g. mice, trackballs, pens or joysticks}; Accessories therefor (digitisers characterised by the transducing means G06F3/041) · CPC title

• G06F3/0346

  with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors · CPC title