Perception and planning collaboration framework for autonomous driving

Patent metadata

Field	Value
Publication number	US-11378956-B2
Application number	US-201815944289-A
Country	US
Kind code	B2
Filing date	Apr 3, 2018
Priority date	Apr 3, 2018
Publication date	Jul 5, 2022
Grant date	Jul 5, 2022

Abstract

A perception module is configured to perceive a driving environment surrounding an autonomous driving vehicle (ADV) based on sensor data, and to generate perception information using various perception models or methods. The perception information describes the perceived driving environment. Based on the perception information, a planning module is configured to plan a trajectory representing a route or a path for a current planning cycle. The ADV is then controlled and driven based on the trajectory. In addition, the planning module determines a critical region (also referred to as a critical area) surrounding the ADV based on the trajectory in view of a current location or position of the ADV. The metadata describing the critical region is transmitted to the perception module via an application programming interface (API) to allow the perception module to generate perception information for a next planning cycle in view of the critical region.

First claim

What is claimed is:

1. A computer-implemented method for operating an autonomous driving vehicle, comprising:
perceiving, by a perception module, a driving environment surrounding an autonomous driving vehicle (ADV) based on sensor data obtained from a plurality of sensors, generating perception information for a current driving cycle using a plurality of perception models;
planning, by a planning module, a trajectory for the current driving cycle based on the perception information for the current driving cycle received from the perception module;
determining a critical region surrounding the ADV based on the trajectory in view of a current location of the ADV, including
determining a driving scenario associated with the ADV based on the trajectory and the current location of the ADV, wherein the driving scenario is determined based on a curvature of the trajectory, a speed and a heading direction of the ADV at different points in time on the trajectory,
performing a lookup operation in a database based on the driving scenario to obtain metadata describing the critical region corresponding to the driving scenario, wherein the lookup operation includes locating a mapping entry matching the driving scenario including driving straight scenario, right turn on intersection scenario, left turn on intersection scenario, lane changing to left scenario, and wherein the driving straight scenario corresponds to a first critical region, the right turn on intersection scenario corresponds to a second critical region, the left turn on intersection scenario corresponds to a third critical region, and the lane changing to left scenario corresponds to a fourth critical region,
determining a polygon defining the critical region surrounding the ADV based on the metadata describing the critical region,
calculating vertexes of the polygon based on a shape of the polygon, wherein the vertexes of the polygon are used to determine a dimension and location of the critical region, and
in response to determining the driving scenario associated with the ADV based on the trajectory and the current location of the ADV, constructing the critical region based on the driving scenario;

controlling the ADV to drive according to the trajectory; and
transmitting metadata describing the critical region to the perception module via an application programming interface (API) such that the perception module generates, within a time limit requirement of a single driving cycle, perception information for a next driving cycle in view of the critical region of the ADV, wherein generating the perception information for the next driving cycle comprises generating first perception information perceiving the critical region using a first perception model of the perception models based on three-dimension sensor data; and
generating second perception information perceiving a remaining area other than the critical region using a second perception model of the perception model based on two-dimension sensor data, wherein the first perception information perceiving the critical region describes the driving environment in a higher resolution than the second perception information perceiving the remaining area other than the critical region, and wherein the perception module consumes more processing resources to generate the first perception information than the second perception information.

2. The method of claim 1 , wherein the critical region surrounding the ADV includes one or more areas that the ADV may interfere with other traffic in the next driving cycle.

3. The method of claim 1 , further comprising constructing the critical region in view of the current location of the ADV based on the metadata describing the critical region of the driving scenario.

4. The method of claim 1 , wherein the database comprises a plurality of database entries, wherein each database entry of the plurality of database entries maps a particular driving scenario to a set of metadata describing one or more rules to define a polygon representing the critical region.

5. The method of claim 1 , further comprising:
determining one of a plurality of the perception models based on the critical region and the remaining area other than the critical region for the next driving cycle.

6. The method of claim 5 , wherein the plurality of the perception models include the first perception model, the second perception model, and a third perception model, wherein the third perception model is a hybrid mode using a combination of different perception models.

7. The method of claim 1 , wherein the metadata of the critical region includes a set of rules to determine the critical region based on the driving environment at a point in time.

8. The method of claim 7 , wherein the driving environment includes a lane configuration and size.

9. The method of claim 7 , wherein the driving environment includes a vehicle physical size.

10. A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations, the operations comprising:
perceiving, by a perception module, a driving environment surrounding an autonomous driving vehicle (ADV) based on sensor data obtained from a plurality of sensors, generating perception information for a current driving cycle using a plurality of perception models;
planning, by a planning module, a trajectory for the current driving cycle based on the perception information for the current driving cycle received from the perception module;
determining a critical region surrounding the ADV based on the trajectory in view of a current location of the ADV, including
determining a driving scenario associated with the ADV based on the trajectory and the current location of the ADV, wherein the driving scenario is determined based on a curvature of the trajectory, a speed and a heading direction of the ADV at different points in time on the trajectory,
performing a lookup operation in a database based on the driving scenario to obtain metadata describing the critical region corresponding to the driving scenario, wherein the lookup operation includes locating a mapping entry matching the driving scenario including driving straight scenario, right turn on intersection scenario, left turn on intersection scenario, lane changing to left scenario, and wherein the driving straight scenario corresponds to a first critical region, the right turn on intersection scenario corresponds to a second critical region, the left turn on intersection scenario corresponds to a third critical region, and the lane changing to left scenario corresponds to a fourth critical region,
determining a polygon defining the critical region surrounding the ADV based on the metadata describing the critical region,
calculating vertexes of the polygon based on a shape of the polygon, wherein the vertexes of the polygon are used to determine a dimension and location of the critical region, and
in response to determining the driving scenario associated with the ADV based on the trajectory and the current location of the ADV, constructing the critical region based on the driving scenario;

controlling the ADV to drive according to the trajectory; and
transmitting metadata describing the critical region to the perception module via an application programming interface (API) such that the perception module generates, within a time limit requirement of a single driving cycle, perception information for a next driving cycle in view of the critical region of the ADV, wherein generating the perception information for the next driving cycle comprises generating first perception information perceiving the critical region using a first perc

Assignees

• Baidu Usa Llc

Inventors

• Zhang Liangliang
• Li Dong
• Hu Jiangtao
• Tao Jiaming
• Jiang Yifei

Classifications

• B60W2554/80

  Spatial relation or speed relative to objects · CPC title

• B60W2554/20

  Static objects · CPC title

• B60W2554/4029

  Pedestrians · CPC title

• B60W60/00274 Primary

  considering possible movement changes · CPC title

• B60W60/0015

  specially adapted for safety · CPC title