Determining vanishing points based on lane lines

Patent metadata

Field	Value
Publication number	US-11227167-B2
Application number	US-201916457557-A
Country	US
Kind code	B2
Filing date	Jun 28, 2019
Priority date	Jun 28, 2019
Publication date	Jan 18, 2022
Grant date	Jan 18, 2022

Abstract

In some implementations, a method is provided. The method includes obtaining an image depicting an environment where an autonomous driving vehicle (ADV) may be located. The image comprises a plurality of line indicators. The plurality of line indicators represent one or more lanes in the environment. The image is part of training data for a neural network. The method also includes determining a plurality of line segments based on the plurality of line indicators. The method further includes determining a vanishing point within the image based on the plurality of line segments. The method further includes updating one or more of the image or metadata associated with the image to indicate a location of the vanishing point within the image.

First claim

What is claimed is:

1. A method, comprising:
obtaining an image depicting an environment where an autonomous driving vehicle (ADV) may be located, wherein:
the image comprises a plurality of line indicators;
the plurality of line indicators represent one or more lanes in the environment;
the image is part of training data for a neural network;

determining a plurality of line segments based on the plurality of line indicators;
determining a vanishing point within the image based on the plurality of line segments, wherein each line segment of the plurality of line segments includes a longest straight line portion of a respective line indicator, and wherein the determining a vanishing point within the image based on the plurality of line segments includes determining the vanishing point within the image by extending each line segment towards a top or upper portion of the image; and
updating one or more of the image or metadata associated with the image to indicate a location of the vanishing point within the image.

2. The method of claim 1 , wherein determining the plurality of line segments comprises:
determining a longest line segment for each line indicator of the plurality of line indicators.

3. The method of claim 2 , wherein a threshold number of points in each line segment are within a threshold distance of a respective line indicator.

4. The method of claim 1 , wherein determining the vanishing point with the image based on the plurality of line segments comprises:
determining a plurality of lines based on the plurality of line segments, wherein:
each line of the plurality of lines corresponds to a line segments of the plurality of line segments; and
each line of the plurality of lines extends from a respective line segment towards a top of the image.

5. The method of claim 4 , wherein determining the vanishing point based on the plurality of line segments comprises:
determining a location where the at least two of the plurality of lines intersect, wherein the vanishing point is determined further based on the location where the plurality of lines intersect.

6. The method of claim 5 , wherein all of the plurality of lines intersect at the location.

7. The method of claim 5 , wherein at least one of the plurality of lines does not intersect with one or more other lines at the location.

8. The method of claim 1 , wherein each line indicator of the plurality of line indicators comprises a curved line or a straight line.

9. The method of claim 1 , further comprising:
obtaining additional images depicting additional environments where the ADV may be located, wherein:
the additional images comprise additional pluralities of line indicators;
the additional pluralities of line indicators represent additional lanes in the additional environments;
the additional images are part of the training data for the neural network;

determining additional pluralities of line segments based on the additional pluralities of line indicators;
determining additional vanishing points within the additional images based on the additional pluralities of line segments; and
updating one or more of the additional images or additional metadata associated with the additional images to indicate additional locations of the vanishing points within the additional images.

10. The method of claim 1 , wherein the plurality of line indicators are based on user input.

11. The method of claim 1 , wherein the plurality of line indicators are based an analysis of the image performed by a computing device.

12. A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations, the operations comprising:
obtaining an image depicting an environment where an autonomous driving vehicle (ADV) may be located, wherein:
the image comprises a plurality of line indicators;
the plurality of line indicators represent one or more lanes in the environment;
the image is part of training data for a neural network;

determining a plurality of line segments based on the plurality of line indicators;
determining a vanishing point within the image based on the plurality of line segments, wherein each line segment of the plurality of line segments includes a longest straight line portion of a respective line indicator, and wherein the determining a vanishing point within the image based on the plurality of line segments includes determining the vanishing point within the image by extending each line segment towards a top or upper portion of the image; and
updating one or more of the image or metadata associated with the image to indicate a location of the vanishing point within the image.

13. The non-transitory machine-readable medium of claim 12 , wherein determining the plurality of line segments comprises:
determining a longest line segment for each line indicator of the plurality of line indicators.

14. The non-transitory machine-readable medium of claim 13 , wherein a threshold number of points in each line segment are within a threshold distance of a respective line indicator.

15. The non-transitory machine-readable medium of claim 12 , wherein determining the vanishing point with the image based on the plurality of line segments comprises:
determining a plurality of lines based on the plurality of line segments, wherein:
each line of the plurality of lines corresponds to a line segments of the plurality of line segments; and
each line of the plurality of lines extends from a respective line segment towards a top of the image.

16. The non-transitory machine-readable medium of claim 15 , wherein determining the vanishing point based on the plurality of line segments comprises:
determining a location where the at least two of the plurality of lines intersect, wherein the vanishing point is determined further based on the location where the plurality of lines intersect.

17. The non-transitory machine-readable medium of claim 16 , wherein all of the plurality of lines intersect at the location.

18. The non-transitory machine-readable medium of claim 16 , at least one of the plurality of lines does not intersect with one or more other lines at the location.

19. A data processing system, comprising:
a processor; and
a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations, the operations including:
obtaining an image depicting an environment where an autonomous driving vehicle (ADV) may be located, wherein:
the image comprises a plurality of line indicators;
the plurality of line indicators represent one or more lanes in the environment;
the image is part of training data for a neural network;

determining a plurality of line segments based on the plurality of line indicators;
determining a vanishing point within the image based on the plurality of line segments, wherein each line segment of the plurality of line segments includes a longest straight line portion of a respective line indicator, and wherein the determining a vanishing point within the image based on the plurality of line segments includes determining the vanishing point within the image by extending each line segment towards a top or upper portion of the image; and
updating one or more of the image or metadata associated with the image to indicate a location of the vanishing point within the image.

20. The non-transitory machine-readable medium of claim 12 , wherein the plurality of li

Assignees

• Baidu Usa Llc

Inventors

• Guo Yuliang
• Choe Tae Eun
• Tsoi Ka Wai
• Chen Guang
• Zhang Weide

Classifications

• G06V10/774

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06T7/73 Primary

  using feature-based methods · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title