Cooperative V2X sensor sharing

Patent metadata

Field	Value
Publication number	US-12100301-B2
Application number	US-202217844978-A
Country	US
Kind code	B2
Filing date	Jun 21, 2022
Priority date	Jun 21, 2022
Publication date	Sep 24, 2024
Grant date	Sep 24, 2024

Abstract

A system for sharing sensor data between a vehicle and a plurality of remote system generally includes a vehicle communication system, a three-dimensional (3D) sensor, and a controller. The controller is programmed to generate an initial 3D point cloud using the 3D sensor, generate an occupancy grid map based on the initial 3D point cloud, and select a producer remote system to provide data for a cell of the occupancy grid map classified as a first blindspot. The controller is further programmed to send a data request to the producer remote system using the vehicle communication system, receive data from the producer remote system including information about the cell of the occupancy grid map classified as the first blindspot, generate a merged 3D point cloud by merging the data received from the producer remote system with the initial 3D point cloud, and identify objects using the merged 3D point cloud.

First claim

What is claimed is:

1. A system for sharing sensor data between a vehicle and a plurality of remote systems, the system comprising:
a vehicle communication system;
a three-dimensional (3D) sensor; and
a controller in electrical communication with the vehicle communication system and the 3D sensor, wherein the controller is programmed to:
generate an initial 3D point cloud of an environment surrounding the vehicle using the 3D sensor, wherein each point in the initial 3D point cloud represents a location of a surface in the environment and has an x, y, and z coordinate relative to the vehicle;
generate an occupancy grid map based on the initial 3D point cloud including a cell of the occupancy grid map classified as a first blindspot;
select a producer remote system to provide data for the cell of the occupancy grid map classified as the first blindspot;
send a data request to the producer remote system using the vehicle communication system based at least on the cell of the occupancy grid map classified as the first blindspot;
receive data from the producer remote system including information about the cell of the occupancy grid map classified as the first blindspot;
generate a merged 3D point cloud by merging the data received from the producer remote system with the initial 3D point cloud; and
identify objects in the environment surrounding the vehicle using the merged 3D point cloud.

2. The system of claim 1 , wherein to generate the occupancy grid map the controller is further programmed to:
classify each point in the initial 3D point cloud as at least one of a ground point or an object point based at least in part on the x, y, and z coordinate of each point;
divide the environment surrounding the vehicle into a 2D grid of a plurality of cells, each of the plurality of cells defined by a set of x and y coordinates relative to the vehicle;
determine a quantity of ground points having x and y coordinates within the set of x and y coordinates for each of the plurality of cells in the 2D grid;
determine a quantity of object points having x and y coordinates within the set of x and y coordinates for each of the plurality of cells in the 2D grid;
classify each of the plurality of cells in the 2D grid as a drivable cell, an undrivable cell, or a blindspot cell based at least in part on the quantity of ground points and the quantity of object points having x and y coordinates within the set of x and y coordinates for each of the plurality of cells in the 2D grid; and
define the occupancy grid map as the 2D grid of the plurality of cells.

3. The system of claim 2 , wherein to select the producer remote system, the controller is further programmed to:
receive, using the vehicle communication system, a plurality of remote occupancy grid maps from the plurality of remote systems, each of the plurality of remote occupancy grid maps including a 2D grid of a plurality of remote cells, each of the plurality of remote cells classified as at least one of a remote drivable cell, a remote undrivable cell and a remote blindspot cell;
receive, using the vehicle communication system, a plurality of geographical locations corresponding to each of the plurality of remote systems;
identify, for a first blindspot cell of the occupancy grid map, a plurality of potential producer remote systems with information about the first blindspot cell of the occupancy grid map, wherein the plurality of potential producer remote systems is a subset of the plurality of remote systems; and
select the producer remote system from the plurality of potential producer remote systems to provide data for the first blindspot cell of the occupancy grid map.

4. The system of claim 3 further comprising a global navigation satellite system (GNSS) in electrical communication with the controller, wherein to identify the plurality of potential producer remote systems, the controller is further programmed to:
determine a geographical location of the vehicle using the GNSS;
determine a cell location for the first blindspot cell of the occupancy grid map using the geographical location of the vehicle and the set of x and y coordinates defining the first blindspot cell of the occupancy grid map;
determine a plurality of remote cell locations for each remote drivable cell of each of the plurality of remote occupancy grid maps and each remote undrivable cell of each of the plurality of remote occupancy grid maps based on the geographical locations of each of the plurality of remote systems and the set of x and y coordinates defining each of the plurality of remote cells; and
identify the plurality of potential producer remote systems, wherein the plurality of potential producer remote systems includes a subset of the plurality of remote systems, wherein a remote occupancy grid map of each of the plurality of potential producer remote systems includes at least one remote drivable cell or remote undrivable cell, and wherein the remote cell location of the at least one remote drivable cell or remote undrivable cell is the same as the cell location of the first blindspot cell of the occupancy grid map.

5. The system of claim 3 , wherein to send the data request to the producer remote system, the controller is further programmed to:
request data from the producer remote system for at least one remote drivable cell or remote undrivable cell of a remote occupancy grid map.

6. The system of claim 3 , wherein to receive data from the producer remote system, the controller is further programmed to:
receive a remote 3D point cloud from the producer remote system for at least one remote drivable or remote undriveable cell of a remote occupancy grid map.

7. The system of claim 6 , wherein to generate the merged 3D point cloud, the controller is further programmed to:
calculate a time difference between a timestamp of the remote 3D point cloud and a timestamp of the initial 3D point cloud;
adjust the x, y, and z coordinates of the remote 3D point cloud based on a velocity of the producer remote system to compensate for the velocity of the producer remote system in response to determining that the time difference is less than a predetermined staleness threshold;
transform the remote 3D point cloud such that a coordinate origin of the remote 3D point cloud is the same as a coordinate origin of the initial 3D point cloud;
identify a plurality of overlapping points in the remote 3D point cloud that are expected to have the same x, y, and z coordinates as a plurality of reference points in the initial 3D point cloud based on a plurality of remote cell locations of the remote occupancy grid map from the producer remote system and a plurality of cell locations of the occupancy grid map of the vehicle;
determine a percentage of the plurality of overlapping points that have the same x, y, and z coordinates as the plurality of reference points in the initial 3D point cloud; and
combine the transformed remote 3D point cloud with the initial 3D point cloud to generate the merged 3D point cloud in response to determining that the percentage of the plurality of overlapping points having the same x, y, and z coordinates as the reference points in the initial 3D point cloud is greater than or equal to a predetermined validity threshold.

8. The system of claim 3 , wherein to select the producer remote system, the controller is further programmed to:
select the producer remote system from the plurality of potential producer remote systems based at least in part on a distance between the vehicle and each of the plurality of potential producer remote systems.

9. The system of claim 1 , wherein the controller is further programmed to:
continuously broadcast the occupancy grid map using t

Assignees

• Gm Global Tech Operations Llc
• Univ Michigan

Inventors

• Naserian Mohammad
• Bai Fan
• Zhang Xumiao
• Zhu Ruiyang
• Zhang Qingzhao
• Zhu Xiao
• Mao Zhuoqing Morley

Classifications

• G06F18/24

  Classification techniques · CPC title

• G01S17/89

  for mapping or imaging · CPC title

• G01S19/01

  Satellite radio beacon positioning systems transmitting time-stamped messages, e.g. GPS [Global Positioning System], GLONASS [Global Orbiting Navigation Satellite System] or GALILEO · CPC title

• G06T19/20

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G06T2219/2004

  Aligning objects, relative positioning of parts · CPC title