Method and system for processing a plurality of images so as to detect lanes on a road

Patent metadata

Field	Value
Publication number	US-11900696-B2
Application number	US-201917636191-A
Country	US
Kind code	B2
Filing date	Aug 28, 2019
Priority date	Aug 28, 2019
Publication date	Feb 13, 2024
Grant date	Feb 13, 2024

Abstract

A system and a method for processing a plurality of images, each image of the plurality of images being acquired by a respective image acquisition module of a vehicle and each image acquisition module being oriented outwardly with respect to the vehicle, the method comprising:
elaborating a bird's eye view image of surroundings of the vehicle using pixel values of pixels of at least one portion of each image of the plurality of images as pixel values of the bird's eye view image, and performing, on the bird's eye view image, a detection of at least one lane marked on a surface on which the vehicle is and visible on the bird's eye view image.

First claim

The invention claimed is:

1. A method for processing a plurality of images, each image of the plurality of images being acquired by a respective image acquisition module of a vehicle and each image acquisition module being oriented outwardly with respect to the vehicle, the method comprising:
elaborating a bird's eye view image of surroundings of the vehicle using pixel values of pixels of at least one portion of each image of the plurality of images as pixel values of the bird's eye view image,
performing, on the bird's eye view image, a detection of at least one lane marked on a surface on which the vehicle is and visible on the bird's eye view image,
wherein the detection is performed based on an indication of whether individual pixels of the bird's eye view image belong to a lane, and
fitting a curve on the lane by determining the curve that minimizes a distance between pixels that belong to the lane and pixels that belong to a curve defined using a mathematical representation of the curve.

2. The method of claim 1 , wherein the detection of lanes is performed using a neural network.

3. The method of claim 1 , wherein elaborating the bird's eye view image of the surroundings of the vehicle comprises using at least one look-up table to associate the pixels of said portions of the image of the plurality of images as pixel values of pixels of the bird's eye view image.

4. The method of claim 3 , wherein elaborating the bird's eye view image of the surroundings of the vehicle comprises using a look-up table for each image of the plurality of image.

5. The method of claim 1 , comprising obtaining position information of the vehicle from at least one sensor different from a camera and taking this position information into account when elaborating the bird's eye view image.

6. The method of claim 1 , comprising performing, on the bird's eye view image, a detection of a plurality of lanes marked on the surface on which the vehicle is and visible on the bird's eye view image.

7. The method of claim 1 , comprising a further post-processing step in which the curve is fitted on each detected lane, and a tracking step of the curve with respect to a previously obtained plurality of images.

8. The method of claim 7 , wherein fitting the curve on each detected lane is performed using a multi-curve fitting approach.

9. The method of claim 1 , comprising a preliminary calibration step to obtain calibration data in which pixels of the at least one portion of each image are associated with pixels of the bird's eye view image.

10. A system for processing a plurality of images, each image of the plurality of images having been acquired by a respective image acquisition module of a vehicle and each image acquisition module being oriented outwardly with respect to the vehicle, the system comprising:
a module configured to elaborate a bird's eye view image of surroundings of the vehicle using pixel values of pixels of at least one portion of each image of the plurality of images as pixel values of the bird's eye view image,
a module configured to perform, on the bird's eye view image, a detection of at least one lane marked on a surface on which the vehicle is and visible on the bird's eye view image,
wherein the detection is performed based on an indication of whether individual pixels of the bird's eye view image belong to a lane, and
a module configured to fit a curve on the lane by determining the curve that minimizes a distance between pixels that belong to the lane and pixels that belong to a curve defined using a mathematical representation of the curve.

11. A vehicle comprising the system of claim 10 and equipped with said image acquisition modules.

12. A non-transitory recording medium readable by a computer and having recorded thereon a computer program including instructions for executing a method for processing a plurality of images, each image of the plurality of images being acquired by a respective image acquisition module of a vehicle and each image acquisition module being oriented outwardly with respect to the vehicle, the method comprising:
elaborating a bird's eye view image of surroundings of the vehicle using pixel values of pixels of at least one portion of each image of the plurality of images as pixel values of the bird's eye view image,
performing, on the bird's eye view image, a detection of at least one lane marked on a surface on which the vehicle is and visible on the bird's eye view image,
wherein the detection is performed based on an indication of whether individual pixels of the bird's eye view image belong to a lane, and
fitting a curve on the lane by determining the curve that minimizes a distance between pixels that belong to the lane and pixels that belong to a curve defined using a mathematical representation of the curve.

Assignees

• Toyota Motor Europe
• Univ Leuven Kath

Inventors

• Tamura Kazuki
• Shimizu Hiroaki
• Proesmans Marc
• Verbiest Frank
• Heylen Jonas
• Dawagne Bruno
• Neven Davy
• Debrabandere Bert
• Van Gool Luc

Classifications

• G06V20/588 Primary

  Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T7/80

  Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration · CPC title

• G06V10/751

  Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching · CPC title

• G06V10/82

  using neural networks · CPC title