Image processing method for detecting objects using relative motion

Patent metadata

Field	Value
Publication number	US-9547795-B2
Application number	US-201214113415-A
Country	US
Kind code	B2
Filing date	Jan 20, 2012
Priority date	Apr 25, 2011
Publication date	Jan 17, 2017
Grant date	Jan 17, 2017

Abstract

An image based obstacle detection method. A camera mounted on a vehicle provides a set of image frames while the vehicle is in motion. The image frames define an image plane having a vertical aspect and a horizontal aspect. The relevancy of an object is determined by (i) selecting first and second feature points from the object that are spaced apart vertically in a first image frame; (ii) tracking the positions of the first and second feature points over at least a second image frame; and (iii) deciding the object to be relevant if the first and second feature points move dissimilar distances in physical space, within a tolerance, and deciding the object to be irrelevant otherwise. The motion of relevant objects is then estimated to determine if any relevant object is likely to become an obstacle to the vehicle.

First claim

The invention claimed is:

1. An obstacle detection method, comprising:
(a) provisioning a vehicle with a camera having an external field of view, the camera having an imager defining an image plane having a vertical aspect and a horizontal aspect, wherein the imager comprises an array of sensing pixels;
(b) acquiring a set of image frames captured by the camera while the vehicle is in motion;
(c) determining an object present in the set of captured image frames by edge detection;
(d) determining a relevancy of the determined object by
(i) selecting first and second feature points of the determined object that are spaced apart vertically in a first captured image frame,
(ii) tracking positions of the first and second feature points of the determined object over at least a second captured image frame, and
(iii) determining that the determined object is a relevant object responsive to a determination that the first and second feature points move dissimilar distances in physical space between the first and second captured image frames and within a tolerance;

(e) estimating motion of determined relevant objects relative to the vehicle to determine if any determined relevant object is likely to become an obstacle to the vehicle;
wherein step (d)(iii) is determined by reference to the image plane and, where the first feature point is vertically lower than the second feature point, and wherein the method includes:
estimating, from the first captured image frame, a physical distance D 1 of the first feature point O 1b to the camera in a selected physical horizontal plane;
estimating, from the first captured image frame, a physical location D T of the second feature point O 1T assuming that the second feature point physically lies at a height T above the selected physical horizontal plane;
estimating, from the second captured image frame, a physical distance D 2 of the first feature point O 2b to the camera in the selected physical horizontal plane, and determining the distance ΔD between D 2 and D 1 ;
estimating a physical location D* T which is equivalent to D T +ΔD, with D* T being located at a height T above the selected physical horizontal plane;
computing a nominal feature point coordinate O* 2t in the image plane corresponding to D* T ; and
determining that the determined object is the relevant object responsive to a determination that the absolute distance between the nominal feature point coordinate O* 2t and the second feature point coordinate in the first captured image frame O 1T is less than the absolute distance between the second feature point coordinate in the second captured image frame O 2T and the second feature point coordinate in the first captured image frame O 1T .

2. A method according to claim 1 , wherein step (d)(iii) is determined by reference to physical space and includes:
determining first and second physical locations along a selected physical horizontal plane corresponding to pixel positions of the first and second feature points in the first captured image frame;
determining third and fourth physical locations along the selected real horizontal plane corresponding to pixel positions of the first and second feature points in the second captured image frame;
determining a first distance Δd 1 between the third and first physical locations and determining a second distance Δd 2 between the fourth and second physical locations; and
determining that the determined object is the relevant object responsive to a determination that the difference between the first distance Δd 1 and the second distance Δd 2 exceeds a threshold, and otherwise determining that the determined object is not a relevant object.

3. A method according to claim 2 , including unwarping a given pixel position to account for lens distortion prior to determining a physical location corresponding to the given pixel position.

4. A method according to claim 3 , wherein the first and second feature points are portions of top and bottom edge contours of the determined object.

5. A method according to claim 1 , wherein the selected physical horizontal plane is a ground plane.

6. A method according to claim 5 , including unwarping a given pixel position to account for lens distortion prior to determining a physical location corresponding to the given pixel position.

7. A method according to claim 6 , wherein the first and second feature points are portions of top and bottom edge contours of the determined object.

8. An obstacle detection system for a vehicle, comprising:
a camera mounted at the vehicle, the camera capturing a set of image frames while the vehicle is in motion, the camera comprising an imager defining an image plane having a vertical aspect and a horizontal aspect, wherein the imager comprises an array of sensing pixels;
a controller coupled to the camera, the controller configured to determine an object in the captured image frames and determine a relevancy of the determined object by
(i) selecting first and second feature points on the determined object that are spaced apart vertically in a first captured image frame,
(ii) tracking positions of the first and second feature points over at least a second captured image frame, and
(iii) determining that the determined object is a relevant object responsive to a determination that the first and second feature points move dissimilar distances in physical space and within a tolerance, and otherwise determining that the determined object is not a relevant object; and

the controller estimating relative motion of determined relevant objects to determine if any relevant object is likely to become an obstacle to the vehicle; and
the controller providing an indication to a driver of the vehicle of determined obstacles;
wherein the controller determines the relevancy of the determined object by reference to the image plane and, where the first feature point is vertically lower than the second feature point, the controller:
estimates, from the first captured image frame, a physical distance D 1 of the first feature point O 1b to the camera in a selected physical horizontal plane;
estimates, from the first captured image frame, a physical location D T of the second feature point O 1T assuming that the second feature point physically lies at a height T above the selected physical horizontal plane;
estimates, from the second captured image frame, a physical distance D 2 of the first feature point O 2b to the camera in the selected physical horizontal plane, and determines the distance ΔD between D 2 and D 1 ;
estimates a physical location D* T which is equivalent to D T +ΔD, with D* T being located at a height T above the selected physical horizontal plane;
computes a nominal feature point coordinate O* 2t in the image plane corresponding to D* T ; and
determines the determined object to be the relevant object responsive to a determination that the absolute distance between the nominal feature point coordinate O* 2t and the second feature point coordinate in the first captured image frame O 1T is less than the absolute distance between the second feature point coordinate in the second captured image frame O 2T and the second feature point coordinate in the first captured image frame O 1T .

9. An obstacle detection system according to claim 8 , wherein the controller determines the relevancy of the determined object by reference to physical space, and wherein the controller is configured to:
determine first and second physical locations along a selected physical horizontal plane corresponding to pixel positions of the first and second feature points in the first captured image frame;
determine third and fourth physical locations along the sele

Assignees

• Gupta Nikhil
• Magna Electronics Inc

Inventors

• Gupta Nikhil

Classifications

• G06V20/58

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title

• G06T7/246 Primary

  using feature-based methods, e.g. the tracking of corners or segments · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• B60R2300/8093

  for obstacle warning · CPC title

• G06T2207/30256

  Lane; Road marking · CPC title