System and method for region of interest window generation for attention based perception

Patent metadata

Field	Value
Publication number	US-11875548-B2
Application number	US-202117382459-A
Country	US
Kind code	B2
Filing date	Jul 22, 2021
Priority date	Jul 22, 2021
Publication date	Jan 16, 2024
Grant date	Jan 16, 2024

Abstract

A system for an attention-based perception includes a camera device configured to provide an image of an operating environment of a vehicle. The system further includes a computerized device monitoring the image, analyzing sensor data to identify a feature in the image as corresponding to an object in the operating environment and assign a score for the feature based upon an identification, a location, or a behavior of the object. The computerized device is further operable to define candidate regions of interest upon the image, correlate the score for the feature to the candidate regions of interest to accrue a total region score, select some of the candidate regions for analysis based upon the total region scores, and analyze the portion of the candidate regions to generate a path of travel output. The system further includes a device controlling the vehicle based upon the output.

First claim

What is claimed is:

1. A system for an attention-based perception in a vehicle, comprising:
a camera device configured to provide an image of an operating environment of the vehicle;
a computerized perception device, operable to:
monitor the image from the camera device;
analyze sensor data to identify at least one feature in the image as corresponding to an object in the operating environment of the vehicle and assign a score for the at least one feature based upon one of an identification of the object, a location of the object or a behavior of the object;
define a plurality of candidate regions of interest upon the image, wherein each of the plurality of candidate regions of interest includes a portion of the image;
for each of the plurality of candidate regions of interest, correlate the score for the at least one feature to the plurality of candidate regions of interest to accrue a total region score;
select a portion of the plurality of candidate regions of interest for analysis based upon the total region score for each of the plurality of candidate regions of interest; and
analyze the portion of the plurality of candidate regions of interest to generate a path of travel output; and

a vehicle control device receiving the path of travel output and controlling the vehicle based upon the path of travel output.

2. The system of claim 1 , wherein analyzing the sensor data includes analyzing data from a plurality of sources; and
wherein assigning the score includes summing a plurality of scores generated by analyzing the data from the plurality of sources.

3. The system of claim 1 , wherein analyzing the sensor data includes analyzing data through a plurality of processes; and
wherein assigning the score includes summing a plurality of scores generated by analyzing the data through the plurality of processes.

4. The system of claim 3 , wherein the plurality of processes include at least one of a maneuver analysis process, wherein an entered navigational route for the vehicle is utilized to identify a road surface in the operating environment, a map analysis process including utilizing a location of the vehicle and three-dimensional map data to determine a geometry of the road surface in the operating environment, an object identification process, a salient feature process, or a depth variance process operable to determine a relative position of a first identified object to a second identified object.

5. The system of claim 1 , wherein the sensor data is the image from the camera device.

6. The system of claim 1 , wherein the sensor data is a light detection and ranging data map provided by a light detection and ranging device.

7. The system of claim 1 , wherein analyzing the sensor data includes utilizing a maneuver analysis process, wherein an entered navigational route for the vehicle is utilized to identify a road surface.

8. The system of claim 1 , wherein analyzing the sensor data includes utilizing a map analysis process including utilizing a location of the vehicle and three-dimensional map data to determine a geometry of a road surface in the operating environment.

9. The system of claim 1 , wherein analyzing the sensor data includes utilizing an object identification process.

10. The system of claim 9 , wherein analyzing the sensor data includes utilizing a scene parsing ambiguity process applying context to identify an object in the operating environment.

11. The system of claim 1 , wherein analyzing the sensor data includes utilizing a salient feature process to determine the score for the at least one feature.

12. The system of claim 1 , wherein analyzing the sensor data includes a depth variance process operable to determine a relative position of a first identified object to a second identified object.

13. The system of claim 1 , wherein the behavior of the object includes one of trajectory of the object or a contextual risk related to the object.

14. The system of claim 1 , wherein the vehicle control device includes an autonomous navigation device operable to control navigation of the vehicle based upon the path of travel output.

15. The system of claim 1 , wherein the vehicle control device includes an object warning device operable to warn an occupant of the vehicle of a presence of the object in the operating environment.

16. A system for an attention-based perception in a vehicle, comprising:
a camera device configured to provide an image of an operating environment of the vehicle;
a computerized perception device, operable to:
monitor the image from the camera device;
analyze sensor data through a plurality of processes to identify at least one feature in the image as corresponding to an object in the operating environment of the vehicle and assign a score for the at least one feature based upon one of an identification of the object, a location of the object or a behavior of the object, wherein assigning the score includes summing a plurality of scores generated by analyzing the sensor data through the plurality of processes, and wherein the plurality of processes include at least one of:
a maneuver analysis process, wherein an entered navigational route for the vehicle is utilized to identify a road surface in the operating environment,
a map analysis process including utilizing a location of the vehicle and three-dimensional map data to determine a geometry of the road surface in the operating environment,
an object identification process,
a salient feature process, or
a depth variance process operable to determine a relative position of a first identified object to a second identified object;

define a plurality of candidate regions of interest upon the image, wherein each of the plurality of candidate regions of interest includes a portion of the image;
for each of the plurality of candidate regions of interest, correlate the score for the at least one feature to the plurality of candidate regions of interest to accrue a total region score;
select a portion of the plurality of candidate regions of interest for analysis based upon the total region score for each of the plurality of candidate regions of interest; and
analyze the portion of the plurality of candidate regions of interest to generate a path of travel output; and

an autonomous navigation device receiving the path of travel output and controlling navigation of the vehicle based upon the path of travel output.

17. The system of claim 16 , wherein analyzing the sensor data includes analyzing data from a plurality of sources.

18. A method for an attention-based perception in a vehicle, comprising:
utilizing a camera device configured to provide an image of an operating environment of the vehicle to provide the image;
within a computerized perception device,
monitoring the image;
analyzing sensor data to identify at least one feature in the image as corresponding to an object in the operating environment of the vehicle and assign a score for the at least one feature based upon one of an identification of the object, a location of the object or a behavior of the object;
defining a plurality of candidate regions of interest upon the image, wherein each of the plurality of candidate regions of interest includes a portion of the image;
for each of the plurality of candidate regions of interest, correlating the score for the at least one feature to the plurality of candidate regions of interest to accrue a total region score;
selecting a portion of the plurality of candidate regions of interest for analysis based upon the total region score f

Assignees

• Gm Global Tech Operations Llc

Inventors

• Tong Wei
• Wang Shige
• Zeng Shuqing

Classifications

• G06V10/25 Primary

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title

• B60W40/06

  Road conditions · CPC title

• B60W50/14

  Means for informing the driver, warning the driver or prompting a driver intervention · CPC title

• B60W60/0011 Primary

  involving control alternatives for a single driving scenario, e.g. planning several paths to avoid obstacles · CPC title

• G01S17/89

  for mapping or imaging · CPC title