Machine learning for event detection and classification in autonomous vehicles

Patent metadata

Field	Value
Publication number	US-10520947-B2
Application number	US-201715469981-A
Country	US
Kind code	B2
Filing date	Mar 27, 2017
Priority date	Mar 27, 2017
Publication date	Dec 31, 2019
Grant date	Dec 31, 2019

Abstract

The present disclosure provides systems and methods for automatic event detection and classification for autonomous vehicles. One example method includes obtaining, by one or more computing devices, vehicle data descriptive of vehicle conditions associated with an autonomous vehicle during an autonomous driving session. The method includes extracting, by the one or more computing devices, a plurality of features from the vehicle data. The method includes determining, by the one or more computing devices using a machine-learned classifier, a classification for each of one or more candidate events based at least in part on one or more of the plurality of features that are respectively associated with the one or more candidate events. The method includes associating, by the one or more computing devices, the classification determined for each of the one or more candidate events with the vehicle data.

First claim

What is claimed is:

1. A computer-implemented method to detect uncomfortable driving events performed by autonomous vehicles, the method comprising:
obtaining, by one or more computing devices, training data that comprises vehicle data logs that were previously collected during previous autonomous vehicle driving sessions, each of the vehicle data logs annotated with event labels that were provided by human passengers during one of the previous autonomous vehicle driving sessions, each event label having a respective label time associated therewith;
analyzing, by the one or more computing devices, each vehicle data log to identify one or more potentially referenced events for each event label, wherein each of the one or more potentially referenced events identified for each event label has an associated event time that is included in a time window prior to the respective label time associated with such event label;
assigning, by the one or more computing devices, each event label to at least one of the one or more potentially referenced events identified for such event label, such that the event label is associated with vehicle data collected at the respective event time which is prior to the respective label time;
after assigning the event labels to the potentially referenced events, training, by the one or more computing devices, a machine-learned classifier using the training data comprising the event labels assigned to the potentially referenced events at the respective event times; and
after training the machine-learned classifier:
obtaining, by the one or more computing devices, vehicle data descriptive of vehicle conditions associated with an autonomous vehicle during an autonomous driving session;
extracting, by the one or more computing devices, a plurality of features from the vehicle data;
determining, by the one or more computing devices using the machine-learned classifier, a classification for each of one or more candidate events based at least in part on one or more of the plurality of features that are respectively associated with the one or more candidate events; and
associating, by the one or more computing devices, the classification determined for each of the one or more candidate events with the vehicle data.

2. The computer-implemented method of claim 1 , wherein extracting, by the one or more computing devices, the plurality of features from the vehicle data comprises performing, by the one or more computing devices, a continuous wavelet transform on the vehicle data to extract the plurality of features.

3. The computer-implemented method of claim 2 , wherein performing, by the one or more computing devices, the continuous wavelet transform comprises performing, by the one or more computing devices, the continuous wavelet transform on the vehicle data with a Mexican Hat wavelet to extract the plurality of features.

4. The computer-implemented method of claim 1 , wherein the plurality of features comprise one or more of: a speed feature, a steering angle feature, and a lateral acceleration feature.

5. The computer-implemented method of claim 1 , wherein extracting, by the one or more computing devices, the plurality of features from the vehicle data comprises:
determining, by the one or more computing devices, a plurality of scale components for each of one or more vehicle data channels; and
computing, by the one or more computing devices for each of the one or more vehicle data channels, a set of statistics that describe the plurality of scale components for such vehicle data channel.

6. The computer-implemented method of claim 5 , wherein computing, by the one or more computing devices for each of the one or more vehicle data channels, the set of statistics that describe the plurality of scale components for such vehicle data channel comprises computing, by the one or more computing devices for each of the plurality of features, one or more of: a mean of the plurality of scale components, a standard deviation of the plurality of scale components, a maximum of the plurality of scale components, a minimum of the plurality of scale components, and a moment of the plurality of scale components.

7. The computer-implemented method of claim 1 , wherein:
extracting, by the one or more computing devices, the plurality of features comprises determining, by the one or more computing devices, a plurality of scale components; and
the method further comprises identifying, by the one or more computing devices, one or more relative peaks associated with the plurality of scale components, each of the one or more relative peaks corresponding to one of the one or more candidate events.

8. The computer-implemented method of claim 1 , wherein determining, by the one or more computing devices using a machine-learned classifier, the classification for each of the one or more candidate events comprises assigning, by the one or more computing devices, at least one of the candidate events to one or more of the following classifications: a high deceleration classification, a high acceleration classification, a juke classification, a jerk classification, and a weaving classification.

9. The computer-implemented method of claim 1 , wherein determining, by the one or more computing devices using the machine-learned classifier, the classification for each of the one or more candidate events comprises, for each candidate event:
inputting, by the one or more computing devices, the one or more features associated with the candidate event into the machine-learned classifier, wherein the machine-learned classifier comprises one or more of a logistic regression classifier, a support vector machine, and a neural network; and
receiving, by the one or more computing devices, the classification for the candidate event as an output of the machine-learned classifier.

10. The computer-implemented method of claim 1 , wherein the one or more computing devices are on-board the autonomous vehicle and the method is iteratively performed in real-time as the autonomous vehicle operates to execute the autonomous driving session.

11. A computer system, comprising:
one or more processors; and
one or more tangible, non-transitory, computer readable media that collectively store:
at least one vehicle data log that was collected during a previous autonomous vehicle driving session, the vehicle data log descriptive of vehicle conditions associated with an autonomous vehicle during the previous autonomous vehicle driving session, the vehicle data log annotated with a plurality of event labels respectively at a plurality of label times; and
instructions that, when executed by the one or more processors, cause the computer system to:
analyze the vehicle data log to identify one or more potentially referenced events for each event label, wherein each of the one or more potentially referenced events identified for each event label has an associated event time that is included in a time window prior to the respective label time associated with such event label;
assign each event label to at least one of the one or more potentially referenced events identified for such event label;
extract, for the event time associated with each potentially referenced event to which one of the event labels has been assigned, one or more features from the vehicle data log;
associate each event label with the one or more features extracted from the vehicle data log for the event time associated with each potentially referenced event to which one of the event labels has been assigned, such that the event label is associated with features extracted from the vehicle data log collected at the respective event time which is prior to the respecti

Assignees

• Uatc Llc

Inventors

• Liu Chenggang
• Bares John Evan

Classifications

• G01C21/26

  specially adapted for navigation in a road network · CPC title

• B60W50/029

  Adapting to failures or work around with other constraints, e.g. circumvention by avoiding use of failed parts · CPC title

• B60W30/095

  Predicting travel path or likelihood of collision · CPC title

• B60W2530/14

  Operations & Transport · mapped topic

• G05D1/0221 Primary

  involving a learning process · CPC title