Learning to simulate

Patent metadata

Field	Value
Publication number	US-11518382-B2
Application number	US-201916696087-A
Country	US
Kind code	B2
Filing date	Nov 26, 2019
Priority date	Sep 26, 2018
Publication date	Dec 6, 2022
Grant date	Dec 6, 2022

Abstract

A method is provided for danger prediction. The method includes generating fully-annotated simulated training data for a machine learning model responsive to receiving a set of computer-selected simulator-adjusting parameters. The method further includes training the machine learning model using reinforcement learning on the fully-annotated simulated training data. The method also includes measuring an accuracy of the trained machine learning model relative to learning a discriminative function for a given task. The discriminative function predicts a given label for a given image from the fully-annotated simulated training data. The method additionally includes adjusting the computer-selected simulator-adjusting parameters and repeating said training and measuring steps responsive to the accuracy being below a threshold accuracy. The method further includes predicting a dangerous condition relative to a motor vehicle and providing a warning to an entity regarding the dangerous condition by applying the trained machine learning model to actual unlabeled data for the vehicle.

First claim

What is claimed is:

1. A computer-implemented method for danger prediction, comprising:
generating, by a hardware processor, fully-annotated simulated training data for a machine learning model responsive to receiving a set of computer-selected simulator-adjusting parameters, the fully-annotated simulated training data including a category for each of pixels included in training images included in the fully-annotated simulated training data;
training, by the hardware processor, the machine learning model using reinforcement learning on the fully-annotated simulated training data;
measuring, by the hardware processor, an accuracy of the trained machine learning model relative to learning a discriminative function for a given task, the discriminative function predicting a given label for a given image from the fully-annotated simulated training data;
adjusting, by the hardware processor, the computer-selected simulator-adjusting parameters and repeating said training and measuring steps responsive to the accuracy being below a threshold accuracy; and
predicting, by the hardware processor, a dangerous condition relative to a motor vehicle and providing a warning to an entity regarding the dangerous condition by applying the trained machine learning model to actual unlabeled data for the motor vehicle.

2. The computer-implemented method of claim 1 , further comprising capturing the actual data using one or more vehicle mounted cameras.

3. The computer-implemented method of claim 1 , wherein said adjusting step is skipped responsive to the accuracy being equal to or greater than a threshold accuracy.

4. The computer-implemented method of claim 3 , wherein a reward is provided responsive to the accuracy being equal to or greater than the threshold accuracy.

5. The computer-implemented method of claim 4 , wherein the reward quantifies an error value, wherein the computer-selected simulator-adjusting parameters are adjusted responsive to a magnitude of the error signal.

6. The computer-implemented method of claim 1 , wherein the parameters are scene parameters that define a probability distribution of a set of scenes.

7. The computer-implemented method of claim 1 , wherein said adjusting step comprising updating a probability distribution of the computer-selected simulator-adjusting parameters.

8. The computer-implemented method of claim 1 , wherein the simulator comprises a scene model implemented as a composition of various scene probability distributions in a graphical model.

9. The computer-implemented method of claim 8 , wherein the graphical model comprises a vehicle path topology comprising a number of lanes, a number of objects in the scene, sidewalks, and landmarks.

10. The computer-implemented method of claim 8 , wherein the graphical model indicates types of the various scene probability distributions.

11. The computer-implemented method of claim 1 , wherein the fully- annotated simulated training data comprises a category for each of pixels comprised in a training image that is comprised in the fully-annotated simulated training data.

12. The computer-implemented method of claim 1 , wherein the fully- annotated simulated training data comprises traffic accident images.

13. The computer-implemented method of claim 1 , wherein the entity is the vehicle and the method further comprises controlling a feature of the vehicle selected from a group consisting of a vehicle speed, a vehicle braking, and vehicle steering, responsive to the dangerous condition being predicted.

14. The computer-implemented method of claim 1 , wherein a number of training epochs in each of a plurality of policy iterations corresponding to said generating, training, and measuring steps is defined as a respective one of the computer-selected simulator- adjusting parameters.

15. The computer-implemented method of claim 1 , wherein a dataset size in each of a plurality of policy iterations corresponding to said generating, training, and measuring steps is defined as a respective one of the computer-selected simulator-adjusting parameters.

16. The computer-implemented method of claim 1 , further comprising selectively choosing between fine-tuning the computer-selected simulator-adjusting parameters and estimating the updated computer-selected simulator-adjusting parameters from the scratch using a random initialization.

17. A computer program product for danger prediction, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method comprising:
generating, by a hardware processor, fully-annotated simulated training data for a machine learning model responsive to receiving a set of computer-selected simulator-adjusting parameters, the fully-annotated simulated training data including a category for each of pixels included in training images included in the fully-annotated simulated training data;
training, by the hardware processor, the machine learning model using reinforcement learning on the fully-annotated simulated training data;
measuring, by the hardware processor, an accuracy of the trained machine learning model relative to learning a discriminative function for a given task, the discriminative function predicting a given label for a given image from the fully-annotated simulated training data;
adjusting, by the hardware processor, the computer-selected simulator-adjusting parameters and repeating said training and measuring steps responsive to the accuracy being below a threshold accuracy; and
predicting, by the hardware processor, a dangerous condition relative to a motor vehicle and providing a warning to an entity regarding the dangerous condition by applying the trained machine learning model to actual unlabeled data for the motor vehicle.

18. The computer program product of claim 17 , wherein said adjusting step is skipped responsive to the accuracy being equal to or greater than a threshold accuracy.

19. The computer program product of claim 18 , wherein a reward is provided responsive to the accuracy being equal to or greater than the threshold accuracy.

20. A computer processing system for determining command-to- process correspondence, comprising:
a memory device including program code stored thereon;
a hardware processor, operatively coupled to the memory device, and configured to run the program code stored on the memory device to
generate fully-annotated simulated training data for a machine learning model responsive to receiving a set of computer-selected simulator-adjusting parameters, the fully-annotated simulated training data including a category for each of pixels included in training images included in the fully-annotated simulated training data;
train the machine learning model using reinforcement learning on the fully- annotated simulated training data;
measure an accuracy of the trained machine learning model relative to learning a discriminative function for a given task, the discriminative function predicting a given label for a given image from the fully-annotated simulated training data;
adjust the computer-selected simulator-adjusting parameters and repeating said training and measuring steps responsive to the accuracy being below a threshold accuracy; and predict a dangerous condition relative to a motor vehicle and providing a warning to an entity regarding the dangerous condition by applying the trained machine learning model to actual unlab

Assignees

• Nec Lab America Inc
• Nec Corp

Inventors

• Schulter Samuel
• Ruiz Nataniel
• Chandraker Manmohan

Classifications

• B60W60/0015 Primary

  specially adapted for safety · CPC title

• G06V20/58

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title

• G06V10/776

  Validation; Performance evaluation · CPC title

• B60W30/0956 Primary

  the prediction being responsive to traffic or environmental parameters · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title