Determining depth using multiple modulation frequencies

Patent metadata

Field	Value
Publication number	US-11994591-B2
Application number	US-202017115180-A
Country	US
Kind code	B2
Filing date	Dec 8, 2020
Priority date	Dec 8, 2020
Publication date	May 28, 2024
Grant date	May 28, 2024

Abstract

Sensors, including time-of-flight sensors, may be used to detect objects in an environment. In an example, a vehicle may include a time-of-flight sensor that images objects around the vehicle, e.g., so the vehicle can navigate relative to the objects. The sensor may generate first image data at a first configuration and second image data at a second configuration. An estimated depth of an object may be determined from the first image data, and an actual depth of the object may be determined from the second image data, based on the estimated depth. In examples, the first and second configurations have different modulation frequencies such that a nominal maximum depth in the first configuration is greater than the nominal maximum depth in the second configuration.

First claim

What is claimed is:

1. A vehicle comprising:
a time-of-flight sensor;
one or more processors; and
memory storing processor-executable instructions that, when executed by the one or more processors, configure the vehicle to:
configure the time-of-flight sensor in a first configuration;
receive, with the time-of-flight sensor in the first configuration, first depth information generated by the time-of-flight sensor, the first depth information being based, at least in part, on a first modulation frequency;
determine, based on the first depth information, an estimated depth of an object from the time-of-flight sensor, the estimated depth not exceeding a first nominal maximum sensor depth associated with the first modulation frequency;
configure the time-of-flight sensor in a second configuration;
receive, with the time-of-flight sensor in the second configuration, second depth information generated by the time-of-flight sensor, the second depth information being based, at least in part, on a second modulation frequency higher than the first modulation frequency;
determine a plurality of candidate depth measurements based, at least in part, on the second depth information, a first candidate depth of the candidate depth measurements comprising a nominal depth not exceeding a second nominal maximum sensor depth associated with the second modulation frequency and a second candidate depth of the candidate depth measurements comprising a sum of the nominal depth and a multiple of the second nominal maximum sensor depth; and
determine a measured distance of the object from the time-of-flight sensor as one of the plurality of candidate depth measurements closest to the estimated depth.

2. The vehicle of claim 1 , wherein:
the second modulation frequency is selected to provide the second nominal maximum sensor depth less than the first nominal maximum sensor depth associated with the first modulation frequency.

3. The vehicle of claim 2 , wherein the first modulation frequency is equal to or less than 10 MHz and the second modulation frequency is equal to or greater than 15 MHz.

4. The vehicle of claim 1 , wherein the plurality of candidate depth measurements comprises a first depth measurement and a second depth measurement, the second depth measurement comprising a sum of the first depth measurement and the second nominal maximum sensor depth associated with the second modulation frequency.

5. The vehicle of claim 1 , wherein the instructions further configure the vehicle to:
determine, based at least in part on at least one of the measured distance of the object, a position of the object in an environment of the vehicle;
determine a travel path through the environment relative to the object; and
control the vehicle to travel along the travel path.

6. A method comprising:
configuring a time-of-flight sensor in a first configuration, the first configuration having an associated first modulation frequency;
emitting, with the time-of-flight sensor in the first configuration, first emitted light at the first modulation frequency;
receiving, at the time-of-flight sensor, first return light associated with the first emitted light;
determining, based at least in part on the first return light, first depth information about a surface;
determining, based on the first depth information, an estimated depth of the surface from the time-of-flight sensor, the estimated depth not exceeding a first nominal maximum depth of the time-of-flight sensor associated with the first configuration;
configuring the time-of-flight sensor in a second configuration having an associated second modulation frequency higher than the first modulation frequency;
emitting, with the time-of-flight sensor in the second configuration, second emitted light at the second modulation frequency;
receiving, at the time-of-flight sensor, second return light associated with the second emitted light;
determining, based at least in part on the second return light, second data comprising second depth information about the surface;
determining, based at least in part on the second depth information, a plurality of candidate depths of the surface, the plurality of candidate depths comprising a nominal depth not exceeding a second nominal maximum depth of the time-of-flight sensor associated with the second configuration and a first candidate depth comprising a sum of the nominal depth and the second nominal maximum depth; and
determining a measured depth of the surface as one of the plurality of candidate depths closest to the estimated depth.

7. The method of claim 6 , wherein the estimated depth is a sensed depth of the surface determined from the first depth information.

8. The method of claim 6 , wherein the first modulation frequency is selected to provide the first nominal maximum depth of the time-of-flight sensor in the first configuration and the second modulation is selected to provide the second nominal maximum depth of the time-of-flight sensor in the second configuration, the second nominal maximum depth being less than the nominal maximum depth.

9. The method of claim 8 , wherein the determining the measured depth comprises:
determining, based on the second depth information, the plurality of candidate depths;
comparing the plurality of candidate depths to the estimated depth; and
determining the measured depth as the one of the plurality of candidate depths closest to the estimated depth.

10. The method of claim 9 , wherein the plurality of candidate depths comprise a measured depth of the surface determined from the second depth information and a second depth comprising a sum of the measured depth and the second nominal maximum depth of the time-of-flight sensor in the second configuration.

11. The method of claim 10 , wherein the measured depth and the second depth are shorter than the first nominal maximum depth associated with the time-of-flight sensor in the first configuration.

12. The method of claim 6 , wherein the first modulation frequency is equal to or less than 10 MHz and the second modulation frequency is equal to or greater than 15 MHz.

13. The method of claim 12 , wherein the first modulation frequency is from 3 MHz to 5 MHz and the second modulation frequency is from 21 MHz to 24 MHz.

14. The method of claim 6 , further comprising:
determining, based at least in part on the actual depth of the surface, a position of an object in an environment of a vehicle;
determining a trajectory of the vehicle through the environment relative to the object; and
controlling the vehicle to travel along the trajectory.

15. A system comprising:
one or more processors; and
computer-readable storage media storing instructions executable by the one or more processors to perform acts comprising:
causing a time-of-flight sensor to be configured in a first configuration, the first configuration having an associated first modulation frequency;
causing the time-of-flight sensor to emit, with the time-of-flight sensor in the first configuration, first emitted light at the first modulation frequency;
determining, based at least in part on first return light associated with the first emitted light and received at the time-of-flight sensor, first time-of-flight sensor information associated with the first configuration, the first time-of-flight sensor information including first depth information;
determining an estimated distance of a surface based on the first depth information, the estimated distance not exceeding a first nominal maximum depth associated with the first configuration;
causing the time-of-flight sensor to be c

Assignees

• Zoox Inc

Inventors

• Subasingha Subasingha Shaminda
• Chen Yongzhe
• Ferdowsi Mehran
• Holladay Samuel
• Karadeniz Turhan
• Moor Robert Nicholas
• Warga Joseph Patrick
• Waschura Harrison Thomas
• Wilkinson Silas Kogure

Classifications

• G01S17/931 Primary

  of land vehicles · CPC title

• G01S13/931

  of land vehicles · CPC title

• G01S15/931

  of land vehicles · CPC title

• G06T7/521

  from laser ranging, e.g. using interferometry; from the projection of structured light · CPC title

• G06T7/55 Primary

  from multiple images · CPC title