Systems and methods for synchronizing sensor capture

Patent metadata

Field	Value
Publication number	US-10972637-B2
Application number	US-201916298503-A
Country	US
Kind code	B2
Filing date	Mar 11, 2019
Priority date	Mar 13, 2018
Publication date	Apr 6, 2021
Grant date	Apr 6, 2021

Abstract

In one embodiment, a method includes, by a computing device of a first sensor receiving synchronization information from a controller. The synchronization information being generated based on a clock of the controller. The method also includes determining, based on the synchronization information, a first offset between a first clock of the first sensor and the clock of the controller; storing the first offset; and synchronizing, based on the stored first offset and the first clock of the first sensor, a first data capture by the first sensor with a second data capture by a second sensor. The first data capture and the second data capture being requested by the controller.

First claim

What is claimed is:

1. A method comprising, by a computing device of a first sensor:
receiving synchronization information from a controller, the synchronization information being generated based on a clock of the controller;
determining, based on the synchronization information, a first offset between a first clock of the first sensor and the clock of the controller;
storing the first offset at a synchronization table of the first sensor, wherein the synchronization table converts timing information generated relative to the clock of the controller to first timing information generated relative to the first clock of the first sensor; and
performing, based on the stored first offset and the first timing information, a first data capture by the first sensor synchronized with a second data capture by a second sensor, the first data capture and the second data capture being requested by the controller.

2. The method of claim 1 , wherein the synchronizing of the first data capture with the second data capture comprises:
receiving, from the controller, a first trigger signal for triggering data capture, the first trigger signal being generated based on the clock of the controller;
determining, based on the first trigger signal and the stored first offset, a first trigger time relative to the first clock of the first sensor; and
triggering the first sensor to perform the first data capture based on the first trigger time and the first clock of the first sensor.

3. The method of claim 2 , wherein the first trigger signal specifies a scheduled time at which to trigger data capture.

4. The method of claim 3 , further comprising:
transmitting, in response to receiving the synchronization information, the received synchronization information to the controller;
wherein the scheduled time is selected by the controller based on a measured round-trip time of the synchronization information being transmitted from and received by the controller.

5. The method of claim 1 , wherein storing the first offset at the synchronization table comprises associating the first offset with a first entry of a plurality of entries stored at the synchronization table, and wherein the first entry comprises a function to transform the timing information into the first timing information.

6. The method of claim 1 , further comprising:
determining, based on second synchronization information received from the second sensor, a second offset between the first clock of the first sensor and a second clock of the second sensor; and
storing the second offset at the synchronization table of the first sensor, wherein the synchronization table further converts timing information generated relative to the second clock of the second sensor to second timing information generated relative to the first clock of the first sensor.

7. The method of claim 6 , wherein the first offset and the second offset are different.

8. The method of claim 1 , wherein:
the synchronization information comprises a plurality of pulses with an interval that corresponds to a counter incrementation of the clock of the controller; and
the first offset is determined based on a measurement of the plurality of pulses using the first clock of the first sensor.

9. The method of claim 1 , wherein the first offset comprises a frequency offset and a phase offset between the first clock of the first sensor and the clock of the controller.

10. The method of claim 1 , wherein the synchronization information comprises a coded-pulse train associated with an expected coded-pulse train, wherein the determining of the first offset comprises:
comparing the coded-pulse train to a plurality of transformed versions of the expected coded-pulse train; and
determining, based on the step of comparing, which one of the plurality of transformed versions of the expected coded-pulse train corresponds to the coded-pulse train;
wherein the first offset is determined based on a transformation associated with the determined transformed version of the expected coded-pulse train.

11. The method of claim 1 , further comprising:
receiving, after the first data capture is performed, a second synchronization information from the controller;
determining, based on the second synchronization information, a third offset between the first clock of the first sensor and the clock of the controller; and
replacing the stored first offset with the third offset.

12. The method of claim 1 , further comprising a vehicle, wherein the first sensor is disposed onto the vehicle.

13. One or more computer-readable non-transitory storage media embodying software that is operable when executed to cause one or more processors of a first sensor to perform operations comprising:
receiving synchronization information from a controller, the synchronization information being generated based on a clock of the controller;
determining, based on the synchronization information, a first offset between a first clock of the first sensor and the clock of the controller;
storing the first offset at a synchronization table of the first sensor, wherein the synchronization table converts timing information generated relative to the clock of the controller to first timing information generated relative to the first clock of the first sensor; and
performing, based on the stored first offset and the first timing information, a first data capture by the first sensor synchronized with a second data capture by a second sensor, the first data capture and the second data capture being requested by the controller.

14. The media of claim 13 , wherein the software is further operable to:
receive, from the controller, a first trigger signal for triggering data capture, the first trigger signal being generated based on the clock of the controller;
determine, based on the first trigger signal and the stored first offset, a first trigger time relative to the first clock of the first sensor; and
trigger the first sensor to perform the first data capture based on the first trigger time and the first clock of the first sensor.

15. The media of claim 13 , wherein storing the first offset at the synchronization table comprises associating the first offset with a first entry of a plurality of entries stored at the synchronization table, and wherein the first entry comprises a function to transform the timing information into the first timing information.

16. The media of claim 13 , wherein the software is further operable to:
determine, based on second synchronization information received from the second sensor, a second offset between the first clock of the first sensor and a second clock of the second sensor; and
store the second offset at the synchronization table of the first sensor, wherein the synchronization table further converts timing information generated relative to the second clock of the second sensor to second timing information generated relative to the first clock of the first sensor.

17. A system of a first sensor comprising one or more processors that, when executing, are configured to cause the system to:
receive synchronization information from a controller, the synchronization information being generated based on a clock of the controller;
determine, based on the synchronization information, a first offset between a first clock of the first sensor and the clock of the controller;
store the first offset at a synchronization table of the first sensor, wherein the synchronization table converts timing information generated relative to the clock of the controller to first timing information

Assignees

• Lyft Inc

Inventors

• Bangs Corey Frederick
• Briggs Forrest Samuel
• Hansel George James
• Hoffacker James Allen-White
• Lamba Dhruv
• Lee Chi Hoon
• Lu Yi
• Mcginn Brian Thomas
• Sawbridge Phillip

Classifications

• H04N23/66

  Remote control of cameras or camera parts, e.g. by remote control devices · CPC title

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• H04N5/06 Primary

  Generation of synchronising signals · CPC title

• H04N2013/0081

  Depth or disparity estimation from stereoscopic image signals · CPC title

• G01S17/86

  Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders · CPC title