Hazard detection from a camera in a scene with moving shadows

Patent metadata

Field	Value
Publication number	US-9892328-B2
Application number	US-201514737522-A
Country	US
Kind code	B2
Filing date	Jun 12, 2015
Priority date	Jun 18, 2014
Publication date	Feb 13, 2018
Grant date	Feb 13, 2018

Abstract

Computerized methods are performable by a driver assistance system while the host vehicle is moving. The driver assistance system includes a camera connectible to a processor. First and second image frames are captured from the field of view of the camera. Corresponding image points of the road are tracked from the first image frame to the second image frame. Image motion between the corresponding image points of the road is processed to detect a hazard in the road. The corresponding image points are determined to be of a moving shadow cast on the road to avoid a false positive detection of a hazard in the road or the corresponding image points are determined not to be of a moving shadow cast on the road to verify detection of a hazard in the road.

First claim

What is claimed is:

1. A method of processing images captured by a camera mountable in a host vehicle while the vehicle is moving, the method comprising:
capturing a first image frame and a second image frame from a field of view of the camera;
tracking corresponding image points from the first image frame and the second image frame;
detecting a plurality of corresponding suspect hazard image points, from among the tracked image points, based on a comparison of the first image frame to the second image frame;
provisionally classifying the suspect hazard image points as part of a hypothetical static object on a road;
comparing a change in a position of a bottom edge of the hypothetical static object in the first image frame and the second image frame relative to a change in a position of an image point of a road plane in the first image frame and the second image frame; and
rejecting the provisional static object classification if the change in the position of the bottom edge of the hypothetical static object is not attributable to the change in the position of the image point of the road plane or validating the provisional static object classification if the change in the position of the bottom edge of the hypothetical static object is attributable to the change in the position of the image point of the road plane.

2. The method of claim 1 , further comprising:
recognizing in the image frames, images of a moving vehicle; and
associating the corresponding suspect hazard image points with the images of a moving vehicle, thereby determining the corresponding suspect hazard image points is due to a moving shadow cast from the moving vehicle.

3. The method of claim 2 , wherein the associating further comprises:
hypothesizing that an image motion between the corresponding suspect hazard image points is due to a hypothetical moving shadow cast from the moving vehicle;
computing respective distances from the host vehicle to the hypothetical moving shadow and from the host vehicle to the moving vehicle; and
matching the computed distances to the hypothetical moving shadow with the computed distances to the moving vehicle, thereby confirming the hypothesis that the image motion is due to the moving shadow cast from the moving vehicle.

4. The method of claim 2 , wherein the associating further comprises:
hypothesizing that an image motion between the corresponding suspect hazard image points is due to a hypothetical moving shadow cast from the moving vehicle;
determining a relative speed between the host vehicle and the moving vehicle;
calculating image motion of points on the road for the relative speed between the host vehicle and the moving vehicle;
measuring image motion of the hypothetical moving shadow;
comparing the measured image motion of the hypothetical moving shadow to the calculated image motion of points on the road for the relative speed between the host vehicle and the moving vehicle; and
confirming the hypothesis that the suspect hazard image points are of a moving shadow cast from the moving vehicle if the measured image motion matches the calculated image motion.

5. The method of claim 1 , further comprising:
associating an image motion between the corresponding suspect hazard image points with a moving shadow when the image motion of the corresponding suspect hazard image points is determined to be consistent with a static object in the road of height greater than a threshold height.

6. The method of claim 1 , further comprising:
recognizing images of a lane mark in the road; and
associating an image motion between the corresponding suspect hazard image points with a moving shadow when a patch associated with the suspect hazard image points is determined to be superimposed over the images of the lane mark.

7. The method of claim 6 , further comprising:
associating the image motion between the corresponding suspect hazard image points with a moving shadow when the image motion is similar on both sides of the images of the lane mark.

8. The method of claim 1 , further comprising:
recognizing in the image frames an image of an object suspected to cast a hypothetical moving shadow;
drawing a plurality of hypothetical lines in the image frames between image points associated with the hypothetical moving shadow and the image of the object; and
associating the hypothetical moving shadow with the object when a plurality of the hypothetical lines originate from the image of the object.

9. The method of claim 1 , further comprising:
verifying that an image motion between the corresponding suspect hazard image points is due to a moving shadow when the image motion is not aligned with a focus of expansion or when a direction of the image motion changes sign.

10. The method of claim 1 , further comprising:
recognizing in the image frames an object suspected to cast a hypothetical moving shadow; and
performing texture analysis on an image of the object to determine if the image of the object has at least one texture selected from a group consisting of: branch texture and leaf texture.

11. The method of claim 1 , further comprising:
recognizing in the image frames an image of a street lamp and an image of a tree suspected of casting a moving shadow; and
verifying that an image motion between the corresponding suspect hazard image points is due to a moving shadow from the light of the street lamp by constructing hypothetical lines between the image of the street lamp, the image of the tree and the corresponding suspect hazard image points.

12. The method according to claim 1 , further comprising:
determining whether a bottom edge of the hypothetical static object is detected in the first and second image frames.

13. The method according to claim 12 , further comprising:
rejecting the provisional static object classification if no bottom edge of the hypothetical static object is detected.

14. The method according to claim 1 , wherein the bottom edge of the hypothetical static object is a hypothesized bottom edge.

15. The method according to claim 1 , wherein provisionally classifying the suspect hazard image points as part of a hypothetical static object includes:
computing an estimated distance to the hypothetical static object; and
determining an image location of the bottom edge of the hypothetical static object based at least on the estimated distance to the hypothetical static object.

16. The method according to claim 15 , wherein determining the bottom edge of the hypothetical static object includes:
determining an image location of the road plane at the estimated distance.

17. The method according to claim 1 , wherein provisionally classifying the suspect hazard image points of a hypothetical static object includes:
computing a distance and a height of at least one of the corresponding suspect hazard image points assuming that the at least one of the corresponding suspect hazard image points corresponds to a static object on the road; and
determining an image location of the bottom edge of the hypothetical static object based on the computed image height and the computed distance of the least one of the corresponding suspect hazard image points.

18. A driver assistance system mountable in a host vehicle, the driver assistance system including a camera operatively connectible to a processor, the system configured, while the host vehicle is moving, to:
capture a first image frame and a second image frame from a field of view of the camera;
track corresponding image points from the first image frame

Assignees

• Mobileye Vision Technologies Ltd

Inventors

• Stein Gideon
• Dagan Erez

Classifications

• G06T2207/30256

  Lane; Road marking · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T2207/30261

  Obstacle · CPC title

• G06T7/40

  Analysis of texture (depth or shape recovery from texture G06T7/529) · CPC title

• G06K9/00805 Primary

  Physics · mapped topic