Visual odometry for low illumination conditions using fixed light sources

Patent metadata

Field	Value
Publication number	US-10127461-B2
Application number	US-201615235326-A
Country	US
Kind code	B2
Filing date	Aug 12, 2016
Priority date	Aug 12, 2016
Publication date	Nov 13, 2018
Grant date	Nov 13, 2018

Abstract

First and second image data is captured comprising a first and second image, respectively. A fixed light source is identified in each of the first and second images. A first ground plane is determined in the first image data. A first (second) intersection is determined, wherein the first (second) intersection is a point in the first image where a virtual lamp post corresponding to the fixed light source in the first (second) image intersects with the first (second) ground plane. The first image data and the second image data are transformed to provide a first and second inverse perspective map (IPM) comprising a first transformed intersection and a second transformed intersection, respectively. Movement parameters are determined based on the location of the first transformed intersection in the first IPM and the location of the second transformed intersection in the second IPM.

First claim

That which is claimed:

1. A method comprising:
capturing first image data comprising a first image, the first image captured by an imaging device;
identifying a fixed light source from the first image data;
determining a first ground plane in the first image data;
determining a first intersection wherein the first intersection is a point in the first image where a virtual lamp post corresponding to the fixed light source in the first image intersects with the first ground plane, wherein the virtual lamp post extends vertically from the fixed light source to the first ground plane;
capturing second image data comprising a second image, the second image captured by the imaging device, wherein the second image sufficiently overlaps with the first image;
identifying the fixed light source in the second image data;
determining a second ground plane in the second image data;
determining a second intersection, the second intersection is the point in the second image where a virtual lamp post corresponding to the fixed light source in the second image data intersects with the second ground plane;
transforming the first image data and the second image data to obtain a first inverse perspective map comprising a first transformed intersection and a second inverse perspective map comprising a second transformed intersection; and
based at least in part on the first transformed intersection and the second transformed intersection, determining one or more movement parameters related to movement of the imaging device between the capturing of the first image data and the capturing of the second image data.

2. A method according to claim 1 , wherein the movement parameters comprise at least one selected from the group of (a) one or more translation parameters, (b) one or more rotation parameters, (c) a distance traveled, (d) a position, (e) a speed, (f) a velocity, and (g) a heading.

3. A method according to claim 1 , wherein the first ground plane is applied as the second ground plane.

4. A method according to claim 1 , wherein the imaging device is affixed to a mode of transportation at either a fixed height, a known height, or both.

5. A method according to claim 1 , wherein the first and/or second ground plane is determined by computing a homography based at least in part on the first and/or second image data.

6. A method according to claim 1 , wherein the height of the imaging device is determined based on either a length of a lane marking, the distance between consecutive lane markings, width of a lane between a pair of lane markings, or combination thereof based on an image captured by the imaging device.

7. A method according to claim 1 , wherein the first and/or second ground plane is determined using gradients in a color and/or intensity profile of the first or second image data, changes in the color and/or intensity profile of the first or second image data, or a combination thereof.

8. A method according to claim 1 , wherein the first and/or second ground plane is determined based at least in part on one or more local road features identified in at least the first or second image data and another set of captured image data.

9. A method according to claim 1 , wherein the first and/or second ground plane is determined based at least in part on one or more objects identified in the first and/or second image.

10. A method according to claim 1 , wherein a slope of the first and/or second ground plane is inferred based at least in part on an illumination profile caused by a light source associated with a mode of transportation to which the imaging device is affixed reflecting off a surface corresponding to the first or second ground plane.

11. A method according to claim 1 , wherein a slope of the first and/or second ground plane is inferred based on the position of one or more lane markings within the first or second image data.

12. A method according to claim 1 , further comprising
performing a full reconstruction of a surface using dense optical flow techniques for areas illuminated by a light source associated with a mode of transportation to which the imaging device is affixed,
wherein the surface corresponds to the first or second ground plane.

13. A method according to claim 1 , wherein measurements by one or more sensors associated with a mode of transportation to which the imaging device is affixed captured in real- or near real-time with the capturing of the first or second image data are used to determine a slope of the first and/or second ground plane.

14. A method according to claim 1 , wherein the movement parameters are determined in real-time or near real-time with respect to the capturing of the second image data.

15. An apparatus comprising at least one processor and at least one memory storing computer program code, the at least one memory and the computer program code configured to, with the processor, cause the apparatus to at least:
capture first image data comprising a first image, the first image captured by an imaging device;
identify a fixed light source from the first image data;
determine a first ground plane in the first image data;
determine a first intersection wherein the first intersection is a point in the first image where a virtual lamp post corresponding to the fixed light source in the first image intersects with the first ground plane, wherein the virtual lamp post extends vertically from the fixed light source to the first ground plane;
capture second image data comprising a second image, the second image captured by the imaging device, wherein the second image sufficiently overlaps with the first image;
identify the fixed light source in the second image data;
determine a second ground plane in the second image data;
determine a second intersection, the second intersection is the point in the second image where a virtual lamp post corresponding to the fixed light source in the second image data intersects with the second ground plane;
transform the first image data and the second image data to obtain a first inverse perspective map comprising a first transformed intersection and a second inverse perspective map comprising a second transformed intersection; and
based at least in part on the first transformed intersection and the second transformed intersection, determine one or more movement parameters related to movement of the imaging device between the capturing of the first image data and the capturing of the second image data.

16. An apparatus according to claim 15 , wherein the movement parameters are determined in real-time or near real-time with respect to the capturing of the second image data.

17. An apparatus according to claim 15 , wherein to determine the first ground plane, the at least one memory and the computer program code are configured to, with the processor, cause the apparatus to at least compute a homography based at least in part on the first image data.

18. A method according to claim 1 , wherein the movement parameters comprise at least one selected from the group of (a) one or more translation parameters, (b) one or more rotation parameters, (c) a distance traveled, (d) a position, (e) a speed, (f) a velocity, and (g) a heading.

19. A computer program product comprising at least one non-transitory computer-readable storage medium having computer-executable program code instructions stored therein, the computer-executable program code instructions comprising program code instructions configured to:
cause first image data comprising a first image to be captured, the

Assignees

• Here Global Bv

Inventors

• Viswanathan Anirudh

Classifications

• H04N23/56

  provided with illuminating means · CPC title

• G06T7/246 Primary

  using feature-based methods, e.g. the tracking of corners or segments · CPC title

• G06K9/00798 Primary

  Physics · mapped topic

• G06T2207/10024

  Color image · CPC title

• G06T2207/30256

  Lane; Road marking · CPC title