Method and apparatus for indicating lane

Patent metadata

Field	Value
Publication number	US-10685241-B2
Application number	US-201715467190-A
Country	US
Kind code	B2
Filing date	Mar 23, 2017
Priority date	Nov 7, 2016
Publication date	Jun 16, 2020
Grant date	Jun 16, 2020

Abstract

Disclosed are methods and apparatuses for indicating a lane, the method including detecting a lane marking from an input image, acquiring a bird's eye view image by applying an inverse perspective mapping to the detected lane marking, extracting control points corresponding to the lane marking from the bird's eye view image, and indicating a lane by re-projecting the control points to a space of the input image through an inverse transformation.

First claim

What is claimed is:

1. A processor-implemented method of indicating a lane, the method comprising:
detecting a plurality of lane markings from an input image by determining a bounding box for each of the lane markings;
acquiring a bird's eye view image by performing an inverse perspective mapping based on the determined bounding boxes;
extracting control points corresponding to the lane marking from the bird's eye view image; and
indicating a lane by re-projecting the control points to a space of the input image.

2. The method of claim 1 , wherein the detecting of the lane markings comprises detecting the lane markings from the input image using a convolutional neural network (CNN) trained to recognize the lane markings.

3. The method of claim 2 , wherein the CNN is trained on lane markings of various road surface views.

4. The method of claim 2 , wherein the CNN is trained to identify the bounding boxes of the lane markings to be detected from the input image and a type of the lane marking to be detected.

5. The method of claim 1 , wherein the detecting of the lane markings comprises:
dividing a searching area of the input image using a sliding window extending in a horizontal direction; and
detecting the lane markings from the divided searching area.

6. The method of claim 5 , wherein the sliding window comprises at least one of the bounding boxes in a form of a stripe extending in a horizontal direction, and
a size of the stripe is gradually reduced in a direction from bottom to top of the searching area.

7. The method of claim 6 , wherein the acquiring of the bird's eye view image comprises acquiring the bird's eye view image by applying the inverse perspective mapping based on a center of each of the bounding boxes.

8. The method of claim 1 , wherein the extracting of the control points comprises performing a curve approximation on the bird's eye view image and extracting the control points corresponding to the lane markings from the bird's eye view image.

9. The method of claim 8 , wherein the extracting of the control points comprises:
converting the lane markings into a curve by performing the curve approximation on the bird's eye view image; and
extracting the control points corresponding to the curve.

10. The method of claim 1 , wherein the extracting of the control points comprises performing a cubic Bezier curve approximation on the bird's eye view image through a random sample consensus (RANSAC) spline fitting and extracting the control points corresponding to the lane markings.

11. The method of claim 1 , wherein the extracting of the control points comprises:
determining a straight line tendency of the lane markings by performing a Hough transformation on the bird's eye view image;
establishing a bounding box proportional to a lane distance based on the straight line tendency; and
extracting the control points corresponding to the lane markings by performing a RANSAC spline fitting on the lane markings in an area of the bounding box.

12. The method of claim 1 , wherein the input image comprises at least one of a road view or a road surface view.

13. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the method of claim 1 .

14. The method of claim 1 , wherein the detecting of the lane markings comprises:
dividing at least a portion of the input image into widthwise segments; and
detecting the lane markings from one of the widthwise segments.

15. The method of claim 14 , wherein:
heights of the widthwise segments decrease in a direction from a lower portion of the input image to an upper portion of the image, and
each of the widthwise segments covers a same amount of road distance in the input image.

16. The method of claim 1 , wherein the detecting of the bird's eye view image comprises excluding an area adjacent to at least one of the detected lane markings from the inverse perspective mapping used in the acquiring of the bird's eye view image.

17. The method of claim 1 , further comprising:
obtaining the input image; and
determining a position of the lane based on the bird's eye view image,
wherein the acquiring of the bird's eye view image comprises generating the bird's eye view image based on the input image by selectively performing the inverse perspective mapping, wherein an area of the bird's eye view image that does not corresponding to the lane markings of the input image is excluded from the inverse perspective mapping.

18. A processor-implemented method of indicating a lane, the method comprising:
detecting a lane marking from an input image;
acquiring a bird's eye view image by applying an inverse perspective mapping to the detected lane marking;
extracting control points corresponding to the lane marking from the bird's eye view image; and
indicating a lane by re-projecting the control points to a space of the input image through an inverse transformation,
wherein the detecting of the lane marking comprises:
dividing a searching area of the input image using a sliding window extending in a horizontal direction; and
detecting the lane marking from the divided searching area,

wherein the sliding window comprises a bounding box in a form of a stripe extending in a horizontal direction,
wherein a size of the stripe is gradually reduced in a direction from bottom to top of the searching area, and
wherein the acquiring of the bird's eye view image comprises acquiring the bird's eye view image by applying the inverse perspective mapping based on a center of the bounding box.

19. An apparatus for indicating a lane, the apparatus comprising:
a communication interface configured to receive an input image; and
one or more processors configured to:
detect a plurality of lane markings from the input image by determining a bounding box for each of the lane markings,
extract control points corresponding to the lane marking from a bird's eye view image acquired by performing an inverse perspective mapping based on the determined bounding boxes, and
indicate a lane by re-projecting the control points to a space of the input image.

20. The apparatus of claim 19 , further comprising:
a memory configured to store parameters of a trained convolutional neural network (CNN) trained to recognize the lane markings,
wherein the one or more processors are configured to perform the detection of the lane markings from the input image using a CNN to which the parameters are applied.

21. The apparatus of claim 20 , wherein the trained CNN is trained to identify the bounding boxes of the lane markings to be detected from the input image and a type of the lane marking to be detected.

22. The apparatus of claim 19 , wherein:
the one or more processors are further configured to divide a searching area of the input image using a sliding window extending in a horizontal direction and to detect the lane markings from the divided searching area, and
the sliding window comprises at least one of the bounding boxes in a form of a stripe extending in a horizontal direction.

23. The apparatus of claim 19 , wherein the one or more processors are further configured to perform a curve approximation on the bird's eye view image, to convert the lane markings into a curve, and to extract the plurality of control points corresponding to the curve.

24. The apparatus of claim 19 , wherein the one o

Assignees

• Samsung Electronics Co Ltd

Inventors

• Kang Nahyup
• Sung Young Hun
• Lee Keechang

Classifications

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2210/12

  Bounding box · CPC title

• G06V10/753

  Transform-based matching, e.g. Hough transform · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V20/582

  of traffic signs · CPC title