Operation of a vehicle using motion planning with machine learning

Patent metadata

Field	Value
Publication number	US-11899464-B2
Application number	US-201916704366-A
Country	US
Kind code	B2
Filing date	Dec 5, 2019
Priority date	Dec 18, 2018
Publication date	Feb 13, 2024
Grant date	Feb 13, 2024

Abstract

Techniques for operation of a vehicle using machine learning with motion planning include storing, using one or more processors of a vehicle located within an environment, a plurality of constraints for operating the vehicle within the environment. One or more sensors of the vehicle receive sensor data describing the environment. The one or more processors extract a feature vector from the stored plurality of constraints and the received sensor data. The feature vector includes a first feature describing an object located within the environment. A machine learning circuit of the vehicle is used to generate a first motion segment based on the feature vector. A number of violations of the stored plurality of constraints is below a threshold. The one or more processors operate the vehicle in accordance with the generated first motion segment.

First claim

What is claimed is:

1. A method comprising:
storing, using one or more processors of an autonomous vehicle located within an environment, a plurality of constraints for operating the autonomous vehicle within the environment;
receiving, using one or more sensors of the autonomous vehicle, sensor data describing the environment;
receiving, using one or more additional sensors of the autonomous vehicle, additional sensor data describing one or more physical characteristics of a passenger of the autonomous vehicle other than a driver of the autonomous vehicle; and
extracting, using the one or more processors, a feature vector from the stored plurality of constraints, the received sensor data, and the received additional sensor data, wherein the feature vector comprises:
a first feature describing an object located within the environment,
a second feature describing the one or more physical characteristics of the passenger of the autonomous vehicle,
a third feature describing a current operation of the vehicle;

determining, using the one or more processors, a hierarchical ranking of the stored plurality of constraints based on the feature vector, including the first feature, the second feature, and the third feature, wherein a rank of each constraint of the stored plurality of constraints corresponds to a level of navigational safety of the autonomous vehicle;
generating, using the one or more processors, a first motion segment based on (i) the feature vector including the first feature, the second feature, and the third feature and (ii) the hierarchical ranking of the stored plurality of constraints, such that a number of violations of the stored plurality of constraints is below a threshold, wherein the generated first motion segment comprises at least one of:
a trajectory between two spatiotemporal locations of the environment, or
a speed of the autonomous vehicle to avoid a collision of the autonomous vehicle with the object; and

causing, using the one or more processors, the autonomous vehicle to autonomously traverse to a destination in accordance with the generated first motion segment.

2. The method of claim 1 , wherein the generated first motion segment further comprises a directional orientation of the autonomous vehicle to avoid a collision with the object.

3. The method of claim 1 , wherein the first feature of the extracted feature vector comprises at least one of:
a spatiotemporal location of the object;
a speed of the object; or
a directional orientation of the object.

4. The method of claim 1 , wherein:
a fourth feature of the extracted feature vector comprises at least one of a maximum speed of the autonomous vehicle, a maximum acceleration of the autonomous vehicle, or a maximum jerk of the autonomous vehicle; and
the at least one of the maximum speed, the maximum acceleration, or the maximum jerk correspond to a level of passenger comfort measured by one or more passenger sensors of the autonomous vehicle.

5. The method of claim 1 , wherein a fifth feature of the extracted feature vector represents whether the causing of the autonomous vehicle to autonomously transverse to the destination in accordance with the first motion segment causes a traffic light violation.

6. The method of claim 1 , further comprising:
aggregating, using the one or more processors, a plurality of features of the extracted feature vector into a motion planning graph, wherein:
the motion planning graph comprises a plurality of edges; and
each edge of the plurality of edges corresponds to a motion segment of the received plurality of motion segments.

7. The method of claim 6 , wherein the generating, based on the feature vector, of the first motion segment comprises:
selecting, using the one or more processors, the first motion segment corresponding to a first edge of the plurality of edges over a second motion segment corresponding to a second edge of the plurality of edges, wherein:
causing the autonomous vehicle to autonomously traverse to the destination in accordance with the first motion segment causes a violation of a first constraint having a higher rank; and
causing the autonomous vehicle to autonomously traverse to the destination in accordance with the second motion segment causes a violation of a second constraint having a lower rank.

8. The method of claim 6 , wherein:
the motion planning graph comprises a minimum-violation motion planning graph; and
each edge of the plurality of edges is associated with a value of an operational metric of a corresponding motion segment.

9. The method of claim 8 , further comprising generating, using a machine learning circuit, the value of the operational metric of each corresponding motion segment of the plurality of edges of the motion planning graph based on the feature vector.

10. The method claim 9 , wherein the generating of the first motion segment comprises identifying, using the machine learning circuit, for each edge of the plurality of edges of the motion planning graph, a likelihood that the causing of the autonomous vehicle to autonomously traverse to the destination in accordance with a corresponding motion segment causes the operational metric to be below the threshold.

11. The method of claim 8 , further comprising sampling, using the one or more processors, the stored plurality of constraints and the received sensor data to generate a third motion segment for operating the autonomous vehicle within the environment, wherein the operating of the autonomous vehicle in accordance with the third motion segment causes the operational metric associated with operating the autonomous vehicle to be below the threshold.

12. The method of claim 8 , wherein:
the operational metric further comprises a weighted aggregate of a number of violations of the stored plurality of constraints; and
each violation of a constraint of the stored plurality of constraints is weighted by a rank of the constraint.

13. The method of claim 1 , wherein the additional sensor data comprises at least one of:
a heart rate of the passenger,
a temperature of the passenger, or
a pupil dilation of the passenger.

14. The method of claim 1 , wherein the additional sensor data comprises at least one of:
a facial expressions of the passenger, or
a skin conductance of the passenger.

15. The method of claim 1 , wherein the additional sensor data comprises a pressure applied to a seat arm rest of the vehicle by the passenger.

16. An autonomous vehicle comprising:
one or more computer processors; and
one or more non-transitory storage media storing instructions which, when executed by the one or more computer processors, cause the one or more computer processors to:
store a plurality of constraints for operating the autonomous vehicle within an environment;
receive, using one or more sensors of the autonomous vehicle, sensor data describing the environment;
receive, using one or more additional sensors of the autonomous vehicle, additional sensor data describing one or more physical characteristics of a passenger of the autonomous vehicle other than a driver of the autonomous vehicle;
extract a feature vector from the stored plurality of constraints, the received sensor data, and the received additional sensor data, wherein the feature vector comprises:
a first feature describing an object located within the environment,
a second feature describing the one or more physical characteristics of the passenger of the autonomous vehicle, and
a third feature describing a current operation of the vehicle;
determining a hierarchical ranking o

Assignees

• Motional Ad Llc

Inventors

• Vora Sourabh
• Beijbom Oscar Olof
• Liu Shih-Yuan
• Wongpiromsarn Tichakorn
• De Francesco Daniele
• Pendleton Scott D

Classifications

• G08G1/165 Primary

  for passive traffic, e.g. including static obstacles, trees · CPC title

• B60W40/02

  related to ambient conditions · CPC title

• G06N3/08

  Learning methods · CPC title

• G05D1/81

  Handing over between on-board automatic and on-board manual control · CPC title

• G05D1/0221 Primary

  involving a learning process · CPC title