Predicting three-dimensional features for autonomous driving

Patent metadata

Field	Value
Publication number	US-2024304003-A1
Application number	US-202418666598-A
Country	US
Kind code	A1
Filing date	May 16, 2024
Priority date	Feb 1, 2019
Publication date	Sep 12, 2024
Grant date	—

Abstract

A processor coupled to memory is configured to receive image data based on an image captured by a camera of a vehicle. The image data is used as a basis of an input to a trained machine learning model trained to predict a three-dimensional trajectory of a machine learning feature. The three-dimensional trajectory of the machine learning feature is provided for automatically controlling the vehicle.

First claim

What is claimed is:

1 . A system, comprising:
one or more processors configured to:
obtain sensor data associated with one or more sensors of a vehicle;
determine a three-dimensional feature associated with the sensor data based on a machine learning model, the machine learning model configured to generate an output representing the three-dimensional feature based on the sensor data; and
determine one or more control signals to control operation of the vehicle based on the three-dimensional feature; and
provide data associated the one or more control signals to cause operation of the vehicle in accordance with the one or more control signals.

2 . The system of claim 1 , wherein the one or more processors configured to determine the one or more control signals are configured to:
determine updates to one or more of: a steering angle of the vehicle; a speed of the vehicle; a rate of acceleration of the vehicle; or a rate of deceleration of the vehicle.

3 . The system of claim 1 , wherein the one or more processors configured to determine the one or more control signals are configured to:
determine the one or more control signals to maintain a position of the vehicle in a lane associated with the vehicle.

4 . The system of claim 1 , wherein the one or more processors configured to determine the one or more control signals are configured to:
determine the one or more control signals to maintain a position of the vehicle in a lane associated with the vehicle relative to other vehicles or obstacles in proximity to the vehicle.

5 . The system of claim 1 , wherein the one or more processors configured to determine the one or more control signals to control operation of the vehicle based on the three-dimensional feature are configured to:
determine a drivable space based on the three-dimensional feature; and determine the one or more control signals based on the drivable space.

6 . The system of claim 1 , wherein the one or more processors configured to determine a three-dimensional feature associated with the sensor data based on a machine learning model are configured to determine a path of a vehicle in an adjacent lane relative to a lane of the vehicle; and
wherein the one or more processors configured to determine the one or more control signals to control operation of the vehicle based on the three-dimensional feature are configured to:
determine the one or more control signals to control operation of the vehicle based on the path of the vehicle in the adjacent lane.

7 . The system of claim 6 , wherein controlling operation of the vehicle based on the path of the vehicle in the adjacent lane involves adjusting a steering angle of the vehicle; a speed of the vehicle; a rate of acceleration of the vehicle; or a rate of deceleration of the vehicle to avoid contact between the vehicle and the vehicle in the adjacent lane.

8 . The system of claim 1 , wherein the one or more processors configured to determine the three-dimensional feature associated with the sensor data based on a machine learning model are configured to:
determine the three-dimensional feature associated with the sensor data based on the machine learning model, wherein the machine learning model is trained based on a training dataset comprising a plurality of time series elements and a determined ground truth corresponding to at least a portion of the plurality of time series elements.

9 . A method, comprising:
obtaining, by at least one processor, sensor data associated with one or more sensors of a vehicle; determining, by the at least one processor, a three-dimensional feature associated with the sensor data based on a machine learning model, the machine learning model configured to generate an output representing the three-dimensional feature based on the sensor data; and determining, by the at least one processor, one or more control signals to control operation of the vehicle based on the three-dimensional feature; and providing, by the at least one processor, data associated the one or more control signals to cause operation of the vehicle in accordance with the one or more control signals.

10 . The method of claim 9 , wherein determining the one or more control signals comprises:
determining, by the at least one processor, updates to one or more of: a steering angle of the vehicle; a speed of the vehicle; a rate of acceleration of the vehicle; or a rate of deceleration of the vehicle.

11 . The method of claim 9 , wherein determining the one or more control signals comprises:
determining, by the at least one processor, the one or more control signals to maintain a position of the vehicle in a lane associated with the vehicle.

12 . The method of claim 9 , wherein determining the one or more control signals comprises:
determining, by the at least one processor, the one or more control signals to maintain a position of the vehicle in a lane associated with the vehicle relative to other vehicles or obstacles in proximity to the vehicle.

13 . The method of claim 9 , wherein determining the one or more control signals to control operation of the vehicle based on the three-dimensional feature comprises:
determining, by the at least one processor, a drivable space based on the three-dimensional feature; and determining, by the at least one processor, the one or more control signals based on the drivable space.

14 . The method of claim 9 , wherein determining the three-dimensional feature associated with the sensor data based on a machine learning model comprises determining, by the at least one processor, a path of a vehicle in an adjacent lane relative to a lane of the vehicle; and
wherein determining the one or more control signals to control operation of the vehicle based on the three-dimensional feature comprises:
determining, by the at least one processor, the one or more control signals to control operation of the vehicle based on the path of the vehicle in the adjacent lane.

15 . The method of claim 14 , wherein controlling operation of the vehicle based on the path of the vehicle in the adjacent lane involves adjusting a steering angle of the vehicle; a speed of the vehicle; a rate of acceleration of the vehicle; or a rate of deceleration of the vehicle to avoid contact between the vehicle and the vehicle in the adjacent lane.

16 . The system of claim 9 , wherein determining the three-dimensional feature associated with the sensor data based on a machine learning model comprises:
determining, by the at least one processor, the three-dimensional feature associated with the sensor data based on the machine learning model, wherein the machine learning model is trained based on a training dataset comprising a plurality of time series elements and a determined ground truth corresponding to at least a portion of the plurality of time series elements.

17 . A non-transitory computer-readable medium storing instructions thereon that, when executed by one or more processors, cause the one or more processors to:
obtain sensor data associated with one or more sensors of a vehicle; determine a three-dimensional feature associated with the sensor data based on a machine learning model, the machine learning model configured to generate an output representing the three-dimensional feature based on the sensor data; and determine one or more control signals to control operation of the vehicle based on the three-dimensional feature; and provide data associated the one or more control signals to cause operation of the vehicle in accordance with the one or m

Assignees

• Tesla Inc

Inventors

• Elluswamy Ashok Kumar
• Bauch Matthew
• Payne Christopher
• Karpathy Andrej
• Shroff Dhaval
• Ramanandan Arvind
• Hakewill James Robert Howard

Classifications

• G06N3/08

  Learning methods · CPC title

• G05D1/617

  Safety or protection, e.g. defining protection zones around obstacles or avoiding hazards (arrangements for controlling the position or course of two or more vehicles for avoiding collisions therebetween G05D1/693; arrangements for reacting to or preventing system or operator failure G05D1/80) · CPC title

• G05D1/249

  from positioning sensors located off-board the vehicle, e.g. from cameras · CPC title

• G05D1/65

  Following a desired speed profile · CPC title

• G06V10/82

  using neural networks · CPC title