Road surface condition guided decision making and prediction

Patent metadata

Field	Value
Publication number	US-12319271-B2
Application number	US-202318207093-A
Country	US
Kind code	B2
Filing date	Jun 7, 2023
Priority date	Jan 21, 2021
Publication date	Jun 3, 2025
Grant date	Jun 3, 2025

Abstract

Among other things, techniques are described for receiving, from at least one sensor of a vehicle, sensor data associated with a surface along a path to be traveled by a vehicle; using a surface classifier to determine a classification of the surface based on the sensor data; determining, based on the classification of the surface, drivability properties of the surface; planning, based on the drivability properties of the surface, a behavior of the vehicle when driving near the surface or on the surface; and controlling the vehicle based on the planned behavior.

First claim

What is claimed is:

1. A system, comprising:
at least one sensor;
at least one computer-readable medium storing computer-executable instructions;
at least one processor configured to communicate with the at least one sensor and to execute the computer-executable instructions, the execution carrying out operations comprising:
receiving surface property feedback comprising sensor measurements captured when a vehicle was near a known surface or driving on the known surface and vehicle behavior feedback comprising information indicative of a vehicle trajectory when the vehicle was near the known surface or driving on the known surface, from a vehicle controller;
receiving known surface information from a shared dynamic surface map, wherein the known surface information comprises known surface properties and a known surface classification of the known surface;
training a machine learning model using the surface property feedback, the vehicle behavior feedback, and the known surface information, wherein the machine learning model is trained to identify a threshold similarity between sensor measurements of a particular surface and the known surface properties of the known surface to classify the particular surface; and
determining, based on an output of the trained machine learning model, a surface classification of the particular surface as the known surface classification based on the threshold similarity.

2. The system of claim 1 , wherein the known surface properties comprise sensor measurements that were previously generated based on the vehicle moving over the known surface, ranges of the sensor measurements, and previously labeled data for road surface conditions.

3. The system of claim 1 , wherein the shared dynamic surface map is a database shared across a fleet of vehicles.

4. The system of claim 1 , wherein the known surface is a temporary surface, and the known surface information from the shared dynamic surface map expires after an amount of time.

5. The system of claim 4 , wherein the amount of time is associated with a type of the known surface.

6. The system of claim 1 , the operations further comprising:
causing a particular vehicle to change a physical operation based on the surface classification when the particular vehicle drives near the particular surface or on the particular surface.

7. The system of claim 1 , wherein the machine learning model is trained to quantify an extent of a particular property of the particular surface, wherein the particular property is one of the known surface properties.

8. The system of claim 1 , wherein training the machine learning model comprises adjusting weights of the machine learning model based on a comparison of an output of the machine learning model with an expected output of the machine learning model.

9. A method comprising:
receiving surface property feedback comprising sensor measurements captured when a vehicle was near a known surface or driving on the known surface and vehicle behavior feedback comprising information indicative of a vehicle trajectory when the vehicle was near the known surface or driving on the known surface, from a vehicle controller;
receiving known surface information from a shared dynamic surface map, wherein the known surface information comprises known surface properties and a known surface classification of the known surface;
training a machine learning model using the surface property feedback, the vehicle behavior feedback, and the known surface information, wherein the machine learning model is trained to identify a threshold similarity between sensor measurements of a particular surface and the known surface properties of the known surface to classify the particular surface;
determining, based on an output of the trained machine learning model, a surface classification of the particular surface as the known surface classification based on the threshold similarity; and
causing a particular vehicle to drive on a road based on the surface classification when the particular vehicle drives near the particular surface or on the particular surface.

10. The method of claim 9 , wherein the known surface properties comprise sensor measurements that were previously generated based on the vehicle moving over the known surface, ranges of the sensor measurements, and previously labeled data for road surface conditions.

11. The method of claim 9 , wherein the shared dynamic surface map is a database shared across a fleet of vehicles.

12. The method of claim 9 , wherein the known surface is a temporary surface, and the known surface information from the shared dynamic surface map expires after an amount of time.

13. The method of claim 12 , wherein the amount of time is associated with a type of the known surface.

14. A non-transitory computer-readable storage medium comprising at least one program for execution by at least one processor of a device, the at least one program comprising instructions which, when executed by the at least one processor, cause the device to perform a method, the method comprising:
receiving surface property feedback comprising sensor measurements captured when a vehicle was near a known surface or driving on the known surface and vehicle behavior feedback comprising information indicative of a vehicle trajectory when the vehicle was near the known surface or driving on the known surface, from a vehicle controller;
receiving known surface information from a shared dynamic surface map, wherein the known surface information comprises known surface properties and a known surface classification of the known surface;
training a machine learning model using the surface property feedback, the vehicle behavior feedback, and the known surface information, wherein the machine learning model is trained to identify a threshold similarity between sensor measurements of a particular surface and the known surface properties of the known surface to classify the particular surface;
determining, based on an output of the trained machine learning model, a surface classification of the particular surface as the known surface classification based on the threshold similarity; and
causing a particular vehicle to drive on a road based on the surface classification when the particular vehicle drives near the particular surface or on the particular surface.

15. The non-transitory computer-readable storage medium of claim 14 , wherein the known surface properties comprise sensor measurements that were previously generated based on the vehicle moving over the known surface, ranges of the sensor measurements, and previously labeled data for road surface conditions.

Assignees

• Motional Ad Llc

Inventors

• Cserna Bence
• Wolff Eric
• Liu Shih-Yuan

Classifications

• B60W2420/408

  Radar; Laser, e.g. lidar · CPC title

• B60W2420/403

  Image sensing, e.g. optical camera · CPC title

• B60W2420/54

  Audio sensitive means, e.g. ultrasound · CPC title

• B60W40/06 Primary

  Road conditions · CPC title

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title