Determining a road surface characteristic

Patent metadata

Field	Value
Publication number	US-10696227-B2
Application number	US-201815868569-A
Country	US
Kind code	B2
Filing date	Jan 11, 2018
Priority date	Jan 12, 2017
Publication date	Jun 30, 2020
Grant date	Jun 30, 2020

Abstract

Systems and methods are provided for determining a road surface characteristic. In one implementation, a system includes at least one processing device programmed to receive, from at least one camera, at least two images representative of an environment of a vehicle; align at least a portion of the at least two images using estimated motion of the vehicle; provide, to a trained system configured to determine a characteristic of the road surface, at least the aligned portions of the at least two images; receive, from the trained system, the determined characteristic of the road surface; and provide, to a vehicle control system, based on at least the determined characteristic of the road surface, control information for changing at least one setting of the vehicle control system.

First claim

What is claimed is:

1. A system for determining an adjustment for a vehicle control system, the system comprising:
at least one processing device programmed to:
receive, from at least one camera, at least two images representative of an environment of a vehicle, the environment including a road surface ahead of the vehicle, where at least one of the two images includes color information;
align at least a portion of the at least two images using estimated motion of the vehicle;
provide, to a trained system configured to determine a characteristic of the road surface, at least the aligned portions of the at least two images, wherein the trained system is configured to determine the characteristic using the aligned portions of the at least two images;
receive, from the trained system, the determined characteristic of the road surface; and
provide, to the vehicle control system, based on at least the determined characteristic of the road surface, control information for changing at least one setting of the vehicle control system.

2. The system of claim 1 , wherein the at least two images have a resolution of at least one pixel per centimeter of the road surface at a distance of at least five meters ahead of the vehicle.

3. The system of claim 1 , wherein the at least two images have a resolution of at least one pixel per centimeter of the road surface at a distance of at least seven meters ahead of the vehicle.

4. The system of claim 1 , wherein the image portions are from consecutive image frames.

5. The system of claim 1 , wherein the image portions are from image frames that are spaced apart from one another by at least a predefined amount.

6. The system according to claim 5 , wherein the image portions correspond to expected locations of the vehicle along the road which are spaced apart by a predefined distance.

7. The system according to claim 6 , wherein the predefined distance is 0.5 meters to 2 meters.

8. The system of claim 1 , wherein at least one of the image portions includes at least one region of the road surface on which at least one tire of the vehicle is predicted to travel.

9. The system of claim 1 , wherein the trained system is included in the vehicle.

10. The system of claim 9 , wherein the at least one processing device is further programmed to update the trained system using update information received from a remote server.

11. The system of claim 10 , wherein the update information includes information determined from analysis of image data and motion data received from a plurality of vehicles.

12. The system of claim 11 , wherein the remote server is configured to train a plurality of weights using at least the image and motion data received from the plurality of vehicles.

13. The system of claim 10 , wherein the remote server includes a neural network.

14. The system of claim 1 , wherein the determined characteristic of the road surface includes a roughness of the road surface.

15. The system of claim 14 , wherein the road surface includes gravel, dust, dirt, cobblestones, bricks, or a smooth surface.

16. The system of claim 1 , wherein the determined characteristic of the road surface includes a presence of a substance on the road surface.

17. The system of claim 16 , wherein the substance includes water, ice, or snow.

18. The system of claim 1 , wherein the determined characteristic of the road surface includes a texture of the road surface.

19. The system of claim 1 , wherein the vehicle control system includes an electronic stability control system.

20. The system of claim 1 , wherein the vehicle control system includes a suspension system.

21. The system of claim 1 , wherein the estimated motion is determined based on an output of at least one sensor of the vehicle, the at least one sensor being different from the at least one camera.

22. A vehicle, the vehicle comprising:
a vehicle control system; and
at least one processing device programmed to:
receive, from at least one camera, at least two images representative of an environment of the vehicle, the environment including a road surf ace ahead of the vehicle, where at least one of the two images includes color information;
align at least a portion of the at least two images using estimated motion of the vehicle;
provide, to a trained system configured to determine a characteristic of the road surface, at least the aligned portions of the at least two images, wherein the trained system is configured to determine the characteristic using the aligned portions of the at least two images;
receive, from the trained system, the determined characteristic of the road surface; and
provide, to the vehicle control system, based on at least the determined characteristic of the road surface, control information for changing at least one setting of the vehicle control system.

23. The vehicle of claim 22 , wherein the vehicle control system includes an electronic stability control system or a suspension system.

24. A method for determining an adjustment for a vehicle control system, the method comprising:
receive, from at least one camera, at least two images representative of an environment of a vehicle, the environment including a road surface ahead of the vehicle, where at least one of the two images includes color information;
align at least a portion of the at least two images using estimated motion of the vehicle;
provide, to a trained system configured to determine a characteristic of the road surface, at least the aligned portions of the at least two images, wherein the trained system is configured to determine the characteristic using the aligned portions of the at least two images;
receive, from the trained system, the determined characteristic of the road surface; and
provide, to the vehicle control system, based on at least the determined characteristic of the road surf ace, control information for changing at least one setting of the vehicle control system.

25. A non-transitory computer-readable medium storing program instructions for executing the method of claim 24 .

Assignees

• Mobileye Vision Technologies Ltd

Inventors

• Stein Gideon

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• B60W40/064

  Degree of grip · CPC title

• B60W10/22

  including control of suspension systems · CPC title