Collaborative perception for autonomous vehicles

Patent metadata

Field	Value
Publication number	US-11514790-B2
Application number	US-202016830548-A
Country	US
Kind code	B2
Filing date	Mar 26, 2020
Priority date	Mar 26, 2020
Publication date	Nov 29, 2022
Grant date	Nov 29, 2022

Abstract

Collaborative perception is based on recognition that a fleet of AVs and stationary infrastructure objects equipped with sensors may be configured to communicate with one another in sharing their sensor data, thus benefiting from collaborative perception, rather than being limited to their individual perception. Three specific scenarios of collaborative perception are disclosed. The first scenario relates to two AVs in the vicinity of one another exchanging complexity scores indicative of their respective environments. The second scenario relates to an AV detecting that it has a blind spot and seeking other AVs or infrastructure objects to provide information indicative of the environment in the blind spot. The third scenario relates to providing infrastructure objects equipped with sensors in appropriate locations so that, when an AV is in the vicinity of such objects, the AV may receive information from their sensors.

First claim

The invention claimed is:

1. A collaborative perception system for a first vehicle, the collaborative perception system comprising:
at least one processor, and at least one memory; and
instructions encoded on the at least one memory that, when executed, instruct the at least one processor to:
receive an indication of a second vehicle determined to be in a vicinity of the first vehicle and determined to have a view of a blind spot of the first vehicle caused by obstruction,
compute a first complexity score (CS) based on sensor data collected by the first vehicle, the first CS indicative of an environment surrounding the first vehicle,
receive from the second vehicle a second CS computed from sensor data collected by the second vehicle, the second CS indicative of an environment surrounding the second vehicle;
wherein each of the first CS and the second CS includes, for the first vehicle and the second vehicle respectively: a number of vulnerable road users, a number of active vehicles, a number of static obstacles, an occupancy of a free-space grid, a percentage of obstructed motion grid, a position of each of one or more vehicles, and a lane direction of each of one or more vehicles;
determine whether a mismatch between the first CS and the second CS exceeds a threshold; and
upon determining that the mismatch exceeds the threshold:
cause the second vehicle to provide, to the first vehicle, sensor data describing the environment surrounding the second vehicle to account for the mismatch, and
determine and execute a driving maneuver based on the sensor data provided by the second vehicle.

2. The collaborative perception system according to claim 1 , wherein, when determined that the mismatch between the first CS and the second CS exceeds a threshold, the processor executing the instructions is further to provide, by the first vehicle, to the second vehicle, sensor data describing an environment surrounding the first vehicle.

3. The collaborative perception system according to claim 1 , wherein, when determined that the mismatch between the first CS and the second CS does not exceed the threshold, the processor executing the instructions is further to maintain a driving strategy of the first vehicle.

4. The collaborative perception system according to claim 1 , wherein the processor executing the instructions is further to request a vehicle management system, to provide to the first vehicle, the indication that the second vehicle is in the vicinity of the first vehicle, and that the second vehicle has the view of the blind spot of the first vehicle.

5. The collaborative perception system according to claim 1 , wherein the processor executing the instructions is further to:
identify, based on sensor data describing the environment surrounding the first vehicle, the blind spot of the first vehicle, and
request a vehicle management system, to provide to the first vehicle, the indication that the second vehicle is in the vicinity of the first vehicle, and that the second vehicle has a view of the identified blind spot of the first vehicle.

6. The collaborative perception system according to claim 1 , wherein the sensor data describing the environment surrounding the second vehicle includes data indicative of one or more of:
a field of view of the second vehicle,
a position of each of one or more objects surrounding the second vehicle,
a velocity of each of the one or more objects surrounding the second vehicle, and
one or more predictions regarding the environment surrounding the second vehicle.

7. The collaborative perception system according to claim 1 , wherein the occupancy of the free-space grid comprises percentage of scene area occupied by road actors' bounding boxes footprints.

8. The collaborative perception system according to claim 1 , wherein the percentage of obstructed motion grid comprises an amount of observable roadway that is occupied or invisible due to obstructions.

9. The collaborative perception system according to claim 1 , wherein:
the position of each of one or more vehicles comprises locations of the vehicles, and
the lane direction of each of one or more vehicles comprises a direction or an identification of a lane in which the vehicles are traveling.

10. A method comprising:
receiving, by a first vehicle, an indication of a second vehicle determined to be in a vicinity of the first vehicle and determined to have a view of a blind spot of the first vehicle caused by obstruction,
computing a first complexity score (CS) based on sensor data collected by the first vehicle, the first CS indicative of an environment surrounding the first vehicle,
receiving from the second vehicle a second CS computed from sensor data collected by the second vehicle, the second CS indicative of an environment surrounding the second vehicle;
wherein each of the first CS and the second CS includes, for the first vehicle and the second vehicle respectively: a number of vulnerable road users, a number of active vehicles, a number of static obstacles, an occupancy of a free-space grid, a percentage of obstructed motion grid, a position of each of one or more vehicles, and a lane direction of each of one or more vehicles;
determining whether a mismatch between the first CS and the second CS exceeds a threshold; and
upon determining the mismatch exceeds the threshold:
causing the second vehicle to provide, to the first vehicle, sensor data describing the environment surrounding the second vehicle to account for the mismatch; and
determining and executing a driving maneuver based on the sensor data provided by the second vehicle.

11. The method of claim 10 , wherein the first vehicle and the second vehicle use a same algorithm for computing the first CS and the second CS respectively.

12. The method of claim 10 , wherein determining whether the mismatch between the first CS and the second CS exceeds the threshold comprises comparing the first CS and the second CS.

13. The method of claim 10 , further comprising sharing an outcome of the determining step with the second vehicle.

14. The method of claim 10 , wherein the sensor data describing the environment surrounding the second vehicle comprises sensor readings of the second vehicle.

15. The method of claim 10 , wherein the sensor data describing the environment surrounding the second vehicle comprises prediction results of processing of sensor readings of the second vehicle.

16. The method of claim 10 , wherein the provision of the sensor data to the first vehicle occurs only upon determining the mismatch exceeds the threshold.

17. The method of claim 10 , further comprising:
identifying, based on sensor data describing the environment surrounding the first vehicle, the blind spot of the first vehicle, and
requesting a vehicle management system, to provide to the first vehicle, the indication that the second vehicle is in the vicinity of the first vehicle, and that the second vehicle has a view of the identified blind spot of the first vehicle.

18. One or more non-transitory computer-readable media having stored thereon executable instructions to instruct a processor to:
receive at a first vehicle an indication of a second vehicle determined to be in a vicinity of the first vehicle and determined to have a view of a blind spot of the first vehicle caused by obstruction,
compute a first complexity score (CS) based on sensor data collected by the first vehicle, the first CS indicative of an environment surrounding the first vehicle,
receive from the second vehicle a second CS co

Assignees

• Gm Cruise Holdings Llc

Inventors

• Grace Nestor
• Plascencia-Vega Diego
• Gidon Dogan
• Elshenawy Mohamed Mostafa

Classifications

• H04W4/46

  for vehicle-to-vehicle communication [V2V] · CPC title

• H04L67/12

  specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks · CPC title

• G08G1/0112

  from the vehicle, e.g. floating car data [FCD] · CPC title

• G08G1/167

  Driving aids for lane monitoring, lane changing, e.g. blind spot detection · CPC title

• G08G1/166

  for active traffic, e.g. moving vehicles, pedestrians, bikes · CPC title