Method of classifying a condition of a road surface

Patent metadata

Field	Value
Publication number	US-2019057261-A1
Application number	US-201715677649-A
Country	US
Kind code	A1
Filing date	Aug 15, 2017
Priority date	Aug 15, 2017
Publication date	Feb 21, 2019
Grant date	—

Abstract

A method of identifying a condition of a road surface includes capturing at least a first image of the road surface with a first camera, and a second image of the road surface with a second camera. The first image and the second image are tiled together to form a combined tile image. A feature vector is extracted from the combined tile image using a convolutional neural network, and a condition of the road surface is determined from the feature vector using a classifier.

First claim

What is claimed is:

1 . A method of identifying a condition of a road surface, the method comprising:
capturing a first image of the road surface with a camera; capturing a second image of the road surface with the camera; tiling the first image and the second image together to form a combined tile image; extracting a feature vector from the combined tile image; and determining a condition of the road surface from the feature vector with a classifier.

2 . The method set forth in claim 1 , further comprising capturing a third image of the road surface with the camera.

3 . The method set forth in claim 2 , wherein the camera includes a first camera, a second camera, and a third camera, and wherein:
capturing the first image of the road surface with the camera is further defined as capturing the first image of the road surface with the first camera; capturing the second image of the road surface with the camera is further defined as capturing the second image of the road surface with the second camera; and capturing the third image of the road surface with the camera is further defined as capturing the third image of the road surface with the third camera.

4 . The method set forth in claim 2 , wherein tiling the first image and the second image to form the combined tile image is further defined as tiling the first image, the second image, and the third image to form the combined tile image.

5 . The method set forth in claim 2 , wherein:
the first image is actively illuminated by a light source; the second image is passively illuminated by ambient light, and is an image of the road surface in a wheel splash region of a vehicle; and the third image is passively illuminated by ambient light, and is an image of the road surface in a region close to a side of the vehicle.

6 . The method set forth in claim 1 , wherein extracting the feature vector from the combined tile image is further defined as extracting the feature vector from the combined tile image using a convolutional neural network.

7 . The method set forth in claim 1 , wherein determining the condition of the road surface from the feature vector with the classifier includes determining the condition of the road surface to be one of a dry road condition, a wet road condition, or a snow covered road condition.

8 . The method set forth in claim 1 , wherein tiling the first image and the second image together to define the combined tile image includes defining a resolution of the first image and a resolution of the second image.

9 . The method set forth in claim 1 , wherein tiling the first image and the second image together to define the combined tile image includes defining an image size of the first image and an image size of the second image.

10 . The method set forth in claim 1 , wherein the first image and the second image are captured simultaneously.

11 . The method set forth in claim 1 , wherein capturing the first image and the second image includes cropping the first image and the second image from a single image to form the first image and the second image respectively.

12 . The method set forth in claim 1 , wherein capturing the first image and the second image includes cropping at least one of the first image and the second image from a respective separate image to form the first image and the second image respectively.

13 . A method of identifying a condition of a road surface, the method comprising:
capturing a first image of the road surface with a first camera, wherein the first image is actively illuminated by a light source; capturing a second image of the road surface with a second camera, wherein the second image is passively illuminated by ambient light, and is an image of the road surface in a wheel splash region of a vehicle; capturing a third image of the road surface with a third camera, wherein the third image is passively illuminated by ambient light, and is an image of the road surface in a region close to a side of the vehicle; tiling the first image, the second image, and the third image together to form a combined tile image; extracting a feature vector from the combined tile image with a convolutional neural network; and determining a condition of the road surface from the feature vector with a classifier.

14 . The method set forth in claim 13 , wherein determining the condition of the road surface from the feature vector with the classifier includes determining the condition of the road surface to be one of a dry road condition, a wet road condition, or a snow covered road condition.

15 . The method set forth in claim 13 , wherein tiling the first image, the second image, and the third image together to define the combined tile image includes defining a resolution of the first image, a resolution of the second image, and a resolution of the third image.

16 . The method set forth in claim 13 , wherein tiling the first image, the second image, and the third image together to define the combined tile image includes defining an image size of the first image, an image size of the second image, and an image size of the third image.

17 . The method set forth in claim 3 wherein the first image, the second image, and the third image are captured simultaneously.

18 . A vehicle comprising:
a body; at least one camera attached to the body and positioned to capture an image of a road surface in a first region relative to the body, and an image of the road surface in a second region relative to the body; a light source attached to the body and positioned to illuminate the road surface in the first region; a computing unit having a processor, a convolutional neural network, a classifier, and a memory having a road surface condition algorithm saved thereon, wherein the processor is operable to execute the road surface condition algorithm to:
capture a first image of the road surface in the first region with the at least one camera, with the first image actively illuminated by the light source;
capture a second image of the road surface in the second region with the at least one camera;
tile the first image and the second image together to form a combined tile image;
extract a feature vector from the combined tile image with the convolutional neural network; and
determine a condition of the road surface from the feature vector with the classifier.

19 . The vehicle set forth in claim 18 , wherein the at least one camera includes a first camera positioned to capture the image of the road surface in the first region, and a second camera positioned to capture the image of the road surface in the second region.

Assignees

• Gm Global Tech Operations Llc

Inventors

• Tong Wei
• Zhao Qingrong
• Zeng Shuqing
• Litkouhi Bakhtiar B

Classifications

• G06V30/19173

  Classification techniques · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/454 Primary

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06V20/56 Primary

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• G06F18/24

  Classification techniques · CPC title