Fused raised pavement marker detection for autonomous driving using lidar and camera

Patent metadata

Field	Value
Publication number	US-2016180177-A1
Application number	US-201414580287-A
Country	US
Kind code	A1
Filing date	Dec 23, 2014
Priority date	Dec 23, 2014
Publication date	Jun 23, 2016
Grant date	—

Abstract

An in-vehicle system for estimating a lane boundary based on raised pavement markers that mark the boundary. The in-vehicle system includes a camera for obtaining image data regarding reflective raised pavement markers and non-reflective raised pavement markers, an image processor for processing frames of image data captured by the camera, a lidar detector for obtaining lidar data regarding reflective raised pavement markers, and a lidar processor for processing frames of lidar data captured by the lidar detector. The image processor generates a first probabilistic model for the lane boundary and the lidar processor generates a second probabilistic model for the lane boundary. The in-vehicle system fuses the first probabilistic model and the second probabilistic model to generate a fused probabilistic model and estimates the lane boundary based on the fused probabilistic model.

First claim

1 . An in-vehicle system for estimating a lane boundary formed of raised pavement markers disposed on a roadway, the system comprising:
a camera for obtaining image data regarding reflective raised pavement markers and non-reflective raised pavement markers; an image processor for processing frames of image data captured by the camera and generating a first probabilistic model for the lane boundary; a lidar detector for obtaining lidar data regarding reflective raised pavement markers; a lidar processor for processing frames of lidar data captured by the lidar detector and generating a second probabilistic model for the lane boundary; and means for fusing the first probabilistic model and the second probabilistic model to generate a fused probabilistic model and for estimating the lane boundary based on the fused probabilistic model.

2 . The in-vehicle system of claim 1 , wherein
the first probabilistic model includes a first mixture of Gaussian distributions; the second probabilistic model includes a second mixture of Gaussian distributions; and the means for fusing the first probabilistic model and the second probabilistic model includes a processor and a non-transitory data storage on which is stored computer code which, when executed on the processor, causes the in-vehicle system to compute a product of the first probabilistic model and the second probabilistic model.

3 . The in-vehicle system of claim 1 , wherein the image processor is in communication with a first non-transitory data storage on which is stored computer code which, when executed on the image processor, causes the image processor to generate the first probabilistic model by:
identifying candidate reflective and non-reflective raised pavement markers; fitting a first spline to the candidate reflective and non-reflective raised pavement markers; segmenting the first spline; and mixing a first set of probability distributions generated based on respective segments of the first spline.

4 . The in-vehicle system of claim 3 , wherein the lidar processor is in communication with a second non-transitory data storage on which is stored computer code which, when executed on the lidar processor, causes the lidar processor to generate the second probabilistic model by:
identifying candidate reflective raised pavement markers; fitting a second spline to the candidate reflective raised pavement markers; segmenting the second spline; and mixing a second set of probability distributions generated based on respective segments of the second spline.

5 . The in-vehicle system of claim 4 , further comprising a database containing statistical information regarding angular trajectories of the roadway, wherein the first non-transitory data storage and the second non-transitory data storage include respective computer code which, when executed, causes the image processor and the lidar processor to fit the first and second splines based on the statistical information stored in the database.

6 . An in-vehicle system for estimating a lane boundary on a roadway, the system comprising:
one or more detectors that capture image data and lidar data of raised pavement markers on the roadway; a processor and a non-transitory data storage on which is stored computer code which, when executed on the processor, causes the in-vehicle system to:
generate a first probabilistic model for the lane boundary based on the image data;
generate a second probabilistic model for the lane boundary based on the lidar data;
fuse the first probabilistic model and the second probabilistic model to generate a fused probabilistic model; and
estimate the lane boundary based on the fused probabilistic model.

7 . The in-vehicle system of claim 6 , wherein the computer code, when executed on the processor, causes the in-vehicle system to generate the first probabilistic model by:
detecting candidate raised pavement markers; tracking the candidate raised pavement markers over time; eliminating false positives among the candidate raised pavement markers; fitting one or more splines to the candidate raised pavement markers; segmenting the one or more splines; and generating a mixture of probability distributions corresponding to respective segments of the one or more splines.

8 . The in-vehicle system of claim 6 , wherein the computer code, when executed on the processor, causes the in-vehicle system to generate the second probabilistic model by:
detecting candidate raised pavement markers; tracking the candidate raised pavement markers over time; eliminating false positives among the candidate raised pavement markers; fitting one or more splines to the candidate raised pavement markers; segmenting the one or more splines; and generating a mixture of probability distributions corresponding to respective segments of the one or more splines.

9 . The in-vehicle system of claim 6 , further comprising
a database containing statistical information regarding angular trajectories of painted lane markings or roadway boundaries, wherein the computer code, when executed on the processor, causes the in-vehicle system to consult the database when generating the first and second probabilistic models.

10 . The in-vehicle system of claim 6 , wherein the computer code, when executed on the processor, causes the in-vehicle system to fuse the first and second probabilistic models through a multiplication operation.

11 . The in-vehicle system of claim 11 , wherein the first probabilistic model and the second probabilistic model comprise respective mixtures of Gaussian distributions.

12 . The in-vehicle system of claim 6 , wherein
the one or more detectors include a camera and a lidar detector; and the computer code, when executed on the processor, causes the in-vehicle system to
detect candidate raised pavement markers;
determine a dynamic detection range for the lidar detector; and
identify candidate raised pavement markers falling outside the dynamic detection range as false positives.

13 . The vehicle system of claim 6 , wherein the dynamic detection range excludes areas where beams of the lidar detector are obstructed.

14 . A method for estimating a lane boundary on a roadway, the method comprising:
receiving image data; receiving lidar data; generating a first probabilistic model for the lane boundary based on the image data; generating a second probabilistic model for the lane boundary based on the lidar data; fusing the first probabilistic model and the second probabilistic model to generate a fused probabilistic model; and estimating the lane boundary based on the fused probabilistic model.

15 . The method of claim 14 , wherein fusing the first probabilistic model and the second probabilistic model comprises multiplying the first probabilistic model and the second probabilistic model.

16 . The method of claim 14 , further comprising:
detecting candidate reflective and non-reflective raised pavement markers in the image data; tracking the candidate reflective and non-reflective raised pavement markers over time; eliminating false positives among the candidate reflective and non-reflective raised pavement markers; fitting one or more splines to the candidate reflective and non-reflective raised pavement markers; segmenting the one or more splines; and generating the first probabilistic model based on the segmented one or more splines.

17 . The method of claim 14 , further comprising:
detecting candidate reflective raised pavement markers in the lidar data; tracking the candid

Assignees

• Volkswagen Ag

Inventors

• Nguyen Tuan Tran
• Lipinski Daniel
• Spehr Jens

Classifications

• G06V10/803

  of input or preprocessed data · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06V20/588

  Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road · CPC title

• G06T7/12 Primary

  Edge-based segmentation · CPC title

• G06F18/2415

  based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate · CPC title