Systems and methods for a configurable sensor system

Patent metadata

Field	Value
Publication number	US-12061263-B2
Application number	US-201916241963-A
Country	US
Kind code	B2
Filing date	Jan 7, 2019
Priority date	Jan 7, 2019
Publication date	Aug 13, 2024
Grant date	Aug 13, 2024

Abstract

The present disclosure relates generally to systems and methods for generating, processing and correlating data from multiple sensors in an autonomous navigation system, and more particularly to the utilization of configurable and dynamic sensor modules within light detection and ranging systems that enable an improved correlation between sensor data as well as configurability and responsiveness of the system to its surrounding environment.

First claim

What is claimed is:

1. A sensor module comprising:
a LiDAR transceiver configured to transmit a plurality of optical signals, detect a corresponding plurality of return optical signals from a plurality of first locations within an environment of the sensor module, process the return optical signals to generate LiDAR data related to one or more first attributes of the plurality of first locations within the environment during a first time period, and transmit the LiDAR data on a first channel;
a thermal sensor configured to generate thermal data related to one or more second attributes of a plurality of second locations within the environment during the first time period and transmit the thermal data on a second channel;
a multiplexer coupled to the first and second channels, the multiplexer configured to receive the LiDAR data from the first channel associated with the LiDAR transceiver and the thermal data from the second channel associated with the thermal sensor, identify first and second portions of the received LiDAR data that are spatially and temporally correlated with respective first and second portions of the thermal data, generate a first unified data packet comprising the first portion of the received LiDAR data and the spatially and temporally correlated first portion of the received thermal data, generate a second unified data packet comprising the second portion of the received LiDAR data and the spatially and temporally correlated second portion of the received thermal data, and output and transmit the first and second unified data packets on a bus;
a demultiplexer coupled to the bus and configured to de-multiplex the first and second unified data packets into homogeneous LiDAR data and thermal data; and
a calibration engine coupled to the demultiplexer, wherein the calibration engine generates transform information based on calibration parameters;
wherein the demultiplexer receives the transform information from the calibration engine and integrates the transform information with the de-multiplexed homogeneous LiDAR data and thermal data into a point cloud format.

2. The sensor module of claim 1 , further comprising a color sensor.

3. The sensor module of claim 1 wherein the LiDAR transceiver and the thermal sensor detect data along a shared axis.

4. The sensor module of claim 1 wherein the LiDAR data and thermal data are correlated using a 1:1 correspondence between measurement data points from the LiDAR transceiver and the thermal sensor.

5. The sensor module of claim 1 further comprising a non-LiDAR sensor configured to generate non-LiDAR data related to one or more third attributes of a plurality of third locations within the environment during the first time period and transmit the non-LiDAR data on a third channel.

6. The sensor module of claim 5 wherein the multiplexer is coupled to the third channel, the multiplexer configured to generate the first and second unified data packets from the LiDAR data, the thermal data, and the non-LiDAR data.

7. The sensor module of claim 6 wherein the non-LiDAR sensor is a color sensor.

8. The sensor module of claim 1 wherein the first and second unified data packets are correlated relative to a single cloud point.

9. The sensor module of claim 1 , wherein the multiplexer is further configured to identify first and second portions of the LiDAR data that are spatially and temporally correlated with respective first and second portions of the thermal data by:
identifying, from the LiDAR data related to a particular attribute of the one or more first attributes of the plurality of first locations within the environment during the first time period, the first portion of the LiDAR data;
identifying, from the thermal data related to the particular attribute of the one or more second attributes of the plurality of second locations within the environment during the first time period, the first portion of the thermal data;
identifying, from the LiDAR data related to a different attribute of the one or more first attributes of the plurality of first locations within the environment during the first time period, the second portion of the LiDAR data; and
identifying, from the thermal data related to the different attribute of the one or more second attributes of the plurality of second locations within the environment during the first time period, the second portion of the thermal data.

10. A sensor system comprising:
a bus;
a sensor module coupled to the bus, the sensor module comprising a first sensor and a second sensor, wherein the first sensor is a lidar sensor and the second sensor is a non-lidar sensor;
a multiplexer coupled between the sensor module and the bus, wherein the first sensor and the second sensor are separately coupled to the multiplexer over different channels, and wherein the multiplexer is configured to correlate lidar data from the first sensor with data from the second sensor and combine the correlated data into a unified data packet that includes both lidar data and data from the second sensor that is spatially and temporarily correlated with the lidar data;
a control unit comprising a data processing element, a demultiplexer and a calibration engine, wherein:
the calibration engine is coupled to the demultiplexer and generates transform information based on calibration parameters;
the demultiplexer is coupled to: (i) receive the unified data packets from the bus, (ii) demultiplex the unified data packets into homogeneous sensor data including lidar data from the first sensor and sensor data from the second sensor, (iii) receive the transform information from the calibration engine, and (iv) integrate the homogeneous sensor data with the transform information; and
the data processing element is configured to generate a point cloud comprising the homogeneous sensor data with the transform information.

11. The sensor system set forth in claim 10 wherein:
the control unit further comprises a calibration engine;
the demultiplexer associates the homogeneous sensor data with a corresponding sensor and provides the homogeneous data to the calibration engine; and
the calibration engine generates transforms based on calibration parameters.

12. The sensor system set forth in claim 11 wherein the control unit further comprises a driver operatively coupled to activate and provide clock triggers to the first and second sensors.

13. The sensor system set forth in claim 10 wherein the data processing element comprises one or more algorithms for computing object detection, velocity estimation and localization to roads and external maps.

14. The sensor system set forth in claim 10 wherein the demultiplexer receives spatial transform information and integrates it with the demultiplexed unified data packet into a point cloud format.

15. The sensor system set forth in claim 10 wherein the second sensor is a thermal sensor.

Assignees

• Velodyne Lidar Usa Inc

Inventors

• Hall David
• Gopalan Anand
• Rekow Matthew
• Nestinger Stephen
• Banas Tyler

Classifications

• G05D1/247

  using signals provided by artificial sources external to the vehicle, e.g. navigation beacons · CPC title

• G01S17/89

  for mapping or imaging · CPC title

• G01S7/4808

  Evaluating distance, position or velocity data · CPC title

• G05D1/024

  in combination with a laser (lasers per se H01S) · CPC title

• G01S17/931 Primary

  of land vehicles · CPC title