System and method for training a machine learning model deployed on a simulation platform

Patent metadata

Field	Value
Publication number	US-11328219-B2
Application number	US-201815952089-A
Country	US
Kind code	B2
Filing date	Apr 12, 2018
Priority date	Apr 12, 2018
Publication date	May 10, 2022
Grant date	May 10, 2022

Abstract

System and method for training a machine learning model are disclosed. In one embodiment, for each of the driving scenarios, responsive to sensor data from one or more sensors of a vehicle and the driving scenario, driving statistics and environment data of the vehicle are collected while the vehicle is driven by a human driver in accordance with the driving scenario. Upon completion of the driving scenario, the driver is requested to select a label for the completed driving scenario and the selected label is stored responsive to the driver selection. Features are extracted from the driving statistics and the environment data based on predetermined criteria. The extracted features include some of the driving statistics and some of the environment data collected at the different points in time during the driving scenario.

First claim

What is claimed is:

1. A computer-implemented method of training a machine learning model for evaluating autonomous driving of an autonomous driving vehicle (ADV), the method comprising:
for each of a plurality of driving scenarios,
collecting driving statistics and environment data of the ADV from one or more sensors of the ADV, while the ADV is driven by a human driver in manual driving mode in accordance with the driving scenario,
upon completion of the driving scenario, displaying a user interface requesting the human driver to select a label for the completed driving scenario, the label indicating unsafe, safe, uncomfortable or comfortable, wherein the human driver selects the label using the user interface,
storing the selected label responsive to the selection of the human driver, and
extracting features from the driving statistics and the environment data based on predetermined criteria deciding to extract information indicating responses of the ADV and obstacle information, the extracted features including some of the driving statistics and some of the environment data collected at different points in time during the driving scenario; and

training a machine learning model to continuously learn driving behaviors and preferences of the human driver under the plurality of driving scenarios using the extracted features and a plurality of labels selected by the human driver for the plurality of driving scenarios;
wherein the trained machine learning model is deployed for performance of autonomous driving simulation that tests a perception and control system of the ADV by using the trained machine learning model to virtually operate a virtual ADV to drive through the driving scenarios under a simulation environment to test the perception and control system of the ADV;
wherein simulation results from the autonomous driving simulation are used to determine whether the perception and control system of the ADV passes or fails.

2. The method of claim 1 , wherein some of the driving statistics includes information indicating responses of the ADV, and some of the environment data includes obstacle information.

3. The method of claim 1 , wherein the extracted features include, for each driving scenario,
identifiers (IDs) that include obstacle IDs representing obstacles in the driving scenario and a vehicle ID representing the ADV,
a timestamp associated with each of the IDs, wherein the timestamp represents a time T i , where i≥0, recorded at the different points in time while the ADV is driven through the driving scenario,
information indicating responses of the ADV captured at the recorded time T i , and
obstacle information captured at the recorded time T i .

4. The method of claim 2 , wherein
the information indicating responses of the ADV includes speed, acceleration, deceleration, and rate of change of acceleration of the ADV, and
the obstacle information includes positions of obstacles.

5. The method of claim 1 , further comprising:
subsequent to collecting driving statistics and environment data of the ADV, storing the collected driving statistics and the environment data remotely onto a server; and
prior to extracting features from the driving statistics and the environment data, retrieving the collected driving statistics and the environment data from the server.

6. A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations, the operations comprising:
for each of a plurality of driving scenarios,
collecting driving statistics and environment data of an autonomous driving vehicle (ADV) from one or more sensors of the ADV, while the ADV is driven by a human driver in manual driving mode in accordance with the driving scenario,
upon completion of the driving scenario, displaying a user interface requesting the driver to select a label for the completed driving scenario, the label indicating unsafe, safe, uncomfortable or comfortable, wherein the human driver selects the label using the user interface,
storing the selected label responsive to the driver selection, and
extracting features from the driving statistics and the environment data based on predetermined criteria deciding to extract information indicating responses of the ADV and obstacle information, the extracted features including some of the driving statistics and some of the environment data collected at different points in time during the driving scenario; and

training a machine learning model to continuously learn driving behaviors and preferences of the human driver under the plurality of driving scenarios using the extracted features and a plurality of labels selected by the human driver for the plurality of driving scenarios;
wherein the trained machine learning model is deployed for performance of autonomous driving simulation that tests a perception and control system of the ADV by using the trained machine learning model to virtually operate a virtual ADV to drive through the driving scenarios under a simulation environment to test the perception and control system of the ADV;
wherein simulation results from the autonomous driving simulation are used to determine whether the perception and control system of the ADV passes or fails.

7. The non-transitory machine-readable medium of claim 6 , wherein some of the driving statistics includes information indicating responses of the ADV, and some of the environment data includes obstacle information.

8. The non-transitory machine-readable medium of claim 6 , wherein the extracted features include, for each driving scenario,
identifiers (IDs) that include obstacle IDs representing obstacles in the driving scenario and a vehicle ID representing the ADV,
a timestamp associated with each of the IDs, wherein the timestamp represents a time T i , where i≥0, recorded at the different points in time while the ADV is driven through the driving scenario,
information indicating responses of the ADV captured at the recorded time T i , and
obstacle information captured at the recorded time T i .

9. The non-transitory machine-readable medium of claim 7 , wherein
the information indicating responses of the ADV includes speed, acceleration, deceleration, and rate of change of acceleration of the ADV, and
the obstacle information includes positions of obstacles.

10. The non-transitory machine-readable medium of claim 6 , wherein the operations further comprise:
subsequent to collecting driving statistics and environment data of the ADV, storing the collected driving statistics and the environment data remotely onto a server; and
prior to extracting features from the driving statistics and the environment data, retrieving the collected driving statistics and the environment data from the server.

11. A data processing system, comprising:
a processor; and
a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations, the operations including
for each of a plurality of driving scenarios,
collecting driving statistics and environment data of an autonomous driving vehicle (ADV) from one or more sensors of the ADV, while the ADV is driven by a human driver in manual driving mode in accordance with the driving scenario,
upon completion of the driving scenario, displaying a user interface requesting the driver to select a label for the completed driving scenario, the label indicating unsafe, safe, uncomfortable or comfortable, wherein the human driver selects the label using the user interface,
storing the selected label responsive to the driver selection, and
extracting features from the d

Assignees

• Baidu Usa Llc

Inventors

• Zhang Liangliang
• Yu Siyang
• Li Dong
• Hu Jiangtao
• Tao Jiaming
• Jiang Yifei

Classifications

• G06F30/20

  Design optimisation, verification or simulation (optimisation, verification or simulation of circuit designs G06F30/30) · CPC title

• G06F30/15 Primary

  Vehicle, aircraft or watercraft design · CPC title

• G07C5/085

  using electronic data carriers · CPC title

• G06N20/00 Primary

  Machine learning · CPC title

• G07C5/008

  communicating information to a remotely located station (transmission systems for measured values G08C) · CPC title