Image sensor processing using a combined image and range measurement system

Patent metadata

Field	Value
Publication number	US-11255952-B2
Application number	US-202016943992-A
Country	US
Kind code	B2
Filing date	Jul 30, 2020
Priority date	May 25, 2018
Publication date	Feb 22, 2022
Grant date	Feb 22, 2022

Abstract

In one embodiment, an apparatus includes a transmitter operable to transmit a first light beam from a light source. The apparatus also includes a receiver operable to receive a plurality of return light beams and direct the plurality of return light beams through a first beam splitter to an imaging sensor and a LiDAR sensor. The imaging sensor may be operable to process a first portion of the return light beams into image profile data, and the LiDAR sensor may be operable to process a second portion of the return light beams into depth profile data. In addition, the first and second portions of the return light beams may be received from a shared field of view.

First claim

What is claimed is:

1. A system comprising:
an image sensor operable to detect light and generate image profile data;
a light detection and ranging (LiDAR) sensor operable to detect light and generate depth profile data;
a transmitter operable to transmit, through a shared field of view, a light beam from a light source to an environment, wherein the transmitter comprises an optical switch operable to switch between transmitting the light beam with different wavelength ranges;
a receiver operable to receive, through the shared field of view, at least one return light beam from the environment; and
a beam splitter operable to:
split the at least one return light beam into a first portion of the at least one return light beam and a second portion of the at least one return light beam, the first portion corresponding to a reflection of the light beam transmitted by the transmitter; and
direct (1) the first portion of the at least one return light beam to the LiDAR sensor to cause a portion of the depth profile data to be generated and (2) the second portion of the at least one return light beam to the image sensor to cause a portion of the image profile data to be generated, wherein the portion of the depth profile data is wholly aligned relative to the portion of the image profile data due to the first portion of the at least one return light beam and the second portion of the at least one return light beam being received through the shared field of view.

2. The system of claim 1 , wherein the LiDAR sensor and the image sensor share the receiver.

3. The system of claim 1 , wherein the system is a solid-state device.

4. The system of claim 1 , wherein the first portion and the second portion aligning includes complete overlap of the field of view between the first portion and the second portion.

5. The system of claim 1 , further comprising a data processor operable to process the portion of the depth profile data and the portion of the image profile data and output a three-dimensional model of the environment.

6. The system of claim 1 , wherein the first portion of the at least one return light beam is associated with infrared light and the second portion of the at least one return light beam is associated with visible light.

7. The system of claim 1 , wherein the portion of the depth profile data and the portion of the image profile data are aligned relative to each other based on at least one of location or time.

8. A system comprising:
a transmitter operable to transmit, through a field of view associated with the transmitter, a light beam from a light source to an environment, wherein the transmitter comprises an optical switch operable to switch between transmitting the light beam with different wavelength ranges;
a receiver operable to receive, through a field of view associated with the receiver, at least one return light beam from the environment, wherein the field of view associated with the receiver correspond to the field of view associated with the transmitter; and
a beam splitter operable to:
split the at least one return light beam into a first portion of the at least one return light beam and a second portion of the at least one return light beam; and
direct (1) the first portion of the at least one return light beam to a first sensor to cause a portion of depth profile data to be generated and (2) the second portion of the at least one return light beam to a second sensor to cause a portion of image profile data to be generated, wherein the portion of the depth profile data and the portion of the image profile data are calibrated with each other due to the first portion of the at least one return light beam and the second portion of the at least one return light beam being received through the field of view associated with the receiver that aligns the first portion and the second portion together.

9. The system of claim 8 , wherein the first sensor and the second sensor share the receiver.

10. The system of claim 8 , wherein the system is a solid-state device.

11. The system of claim 8 , wherein the first portion and the second portion aligning includes complete overlap of the field of view between the first portion and the second portion.

12. The system of claim 8 , further comprising a data processor operable to process the portion of the depth profile data and the portion of the image profile data and output a three-dimensional model of the environment.

13. The system of claim 8 , wherein the first portion of the at least one return light beam is associated with infrared light and the second portion of the at least one return light beam is associated with visible light.

14. The system of claim 8 , wherein the portion of the depth profile data and the portion of the image profile data are calibrated with each other based on at least one of location or time.

15. A method comprising:
causing a light beam to be transmitted by a transmitter, through a shared field of view, from a light source to an environment; and
subsequent to receiving, by a receiver, through the shared field of view, at least one return light beam from the environment, wherein the at least one return light beam is split by a beam splitter into a first portion of the at least one return light beam and a second portion of the at least one return light beam, the first portion corresponding to a reflection of the light beam transmitted by the transmitter, the transmitter comprising an optical switch operable to switch between transmitting the light beam with different wavelength ranges:
generating a portion of depth profile data based on the first portion of the at least one return light beam; and
generating a portion of image profile data based on the second portion of the at least one return light beam, wherein the portion of the depth profile data is wholly aligned relative to the portion of the image profile data due to the first portion of the at least one return light beam and the second portion of the at least one return light beam being received through the shared field of view.

16. The method of claim 15 , wherein the first portion of the at least one return light beam is directed to a light detection and ranging (LiDAR) sensor for generating the portion of the depth profile data, and wherein the second portion of the at least one return light beam is directed to an image sensor for generating the portion of the image profile data.

17. The method of claim 16 , wherein the LiDAR sensor and the image sensor share the receiver.

18. The method of claim 15 , wherein the first portion and the second portion aligning includes complete overlap of the field of view between the first portion and the second portion.

19. The method of claim 15 , further comprising:
processing the portion of the depth profile data and the portion of the image profile data;
and
outputting a three-dimensional model of the environment based on the processed portions of the depth profile data and the image profile data.

20. The method of claim 15 , wherein the first portion of the at least one return light beam is associated with infrared light and the second portion of the at least one return light beam is associated with visible light.

Assignees

• Woven Planet North America Inc

Inventors

• Abari Farzad Cyrus Foroughi
• Clément Romain

Classifications

• H04N23/698

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• G01S17/931

  of land vehicles · CPC title

• G01S17/86

  Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders · CPC title

• G01S17/89

  for mapping or imaging · CPC title

• G01S7/4815 Primary

  using multiple transmitters · CPC title