System for automated lane marking

Patent metadata

Field	Value
Publication number	US-2020210717-A1
Application number	US-201816233989-A
Country	US
Kind code	A1
Filing date	Dec 27, 2018
Priority date	Dec 27, 2018
Publication date	Jul 2, 2020
Grant date	—

Abstract

Systems and processes can automatically identify lane markings within images through the use of a machine learning model. The machine learning model may use a reduced set of data and output an improved estimate of lane markings by applying normalized data or images to the machine learning model. Each image applied to the model can be normalized by, for example, rotating each of the images such that the depicted roads are horizontal or otherwise share the same angle. By aligning disparate images of roads, it is possible to reduce the amount of data applied to the model or to model generation, and to increase the accuracy of the machine learning model. Further, the use of normalized images by the machine learning model enables a reduction in computing resources used to apply data to the machine learning model to, for example, identify lane markings within images.

First claim

What is claimed is:

1 . A computer-implemented method of performing an automated lane marking process on an image of a road network using a machine learning model, the computer-implemented method comprising:
as implemented by a lane detection system comprising one or more hardware processors and configured with specific computer-executable instructions,
accessing a rotated image segment depicting a road segment, wherein the rotated image segment comprises a rotated portion of an image of a road network, and wherein the rotated image segment is rotated to align the road segment with a target axis;
applying the rotated image segment to a machine learning model to obtain a lane marked rotated image segment;
determining a degree of rotation with respect to the target axis of the rotated image segment;
counter rotating the lane marked rotated image segment by the degree of rotation to obtain a lane-marked image segment; and
updating the image of the road network based at least in part on the lane-marked image segment to obtain an updated image.

2 . The computer-implemented method of claim 1 , wherein lane markings on the lane-marked rotated image segment or on the lane-marked image segment are included as part of an overlay layer that provides lane marking data for one or more of: a vehicle routing system, a driver-assist system, or an autonomous vehicle system.

3 . The computer-implemented method of claim 1 , wherein the degree of rotation comprises an amount that a portion of the image of the road network was rotated to obtain the rotated image segment.

4 . The computer-implemented method of claim 1 , wherein the degree of rotation is determined from metadata associated with the rotated image segment.

5 . The computer-implemented method of claim 1 , wherein the image of the road network comprises a map of the road network.

6 . The computer-implemented method of claim 1 , wherein updating the image of the road network comprises replacing a corresponding portion of the image with the lane-marked image segment.

7 . The computer-implemented method of claim 1 , wherein updating the image of the road network comprises applying lane markings from the lane-marked image segment to a corresponding portion of the image.

8 . The computer-implemented method of claim 1 , further comprising storing the updated map at a map repository configured to store mapping data for one or more of: a vehicle routing system, a driver-assist system, or an autonomous vehicle system.

9 . The computer-implemented method of claim 1 , wherein the target axis comprises a horizontal axis.

10 . A system for performing an automated lane marking process on an image of a road network using a machine learning model, the system comprising:
a storage configured to store rotated image segments depicting portions of roads rotated to align with a target axis; and a lane detection system comprising one or more hardware processors configured to:
access from the storage a rotated image segment depicting a road segment, wherein the rotated image segment is a rotated portion of an image of a road network, and wherein the rotated image segment is rotated to align the road segment with the target axis;
input the rotated image segment to a machine learning model to obtain a lane-marked rotated image segment;
determine a degree of rotation with respect to the target axis of the rotated image segment;
counter rotate the lane-marked rotated image segment by the degree of rotation to obtain a lane-marked image segment; and
modify the image of the road network based at least in part on the lane-marked image segment to obtain a modified image.

11 . The system of claim 10 , wherein the lane detection system is further configured to generate an overlay layer that includes lane markings from the lane-marked rotated image segment or the lane-marked image segment, and wherein the overlay layer is provided to one or more of: a vehicle routing system, a driver-assist system, or an autonomous vehicle system.

12 . The system of claim 10 , wherein the degree of rotation comprises an amount that a portion of the image of the road network was rotated to obtain the rotated image segment.

13 . The system of claim 10 , wherein the lane detection system is further configured to modify the image of the road network by replacing a corresponding portion of the image with the lane-marked image segment.

14 . The system of claim 10 , wherein the lane detection system is further configured to modify the image of the road network by applying lane markings from the lane-marked image segment to a corresponding portion of the image.

15 . The system of claim 10 , wherein the lane detection system is further configured to store the modified map at a map repository configured to store mapping data for one or more of: a vehicle routing system, a driver-assist system, or an autonomous vehicle system.

16 . The system of claim 15 , wherein the storage comprises the map repository.

17 . A non-transitory computer-readable storage medium storing computer executable instructions that, when executed by one or more computing devices, configure the one or more computing devices to perform operations comprising:
obtaining a rotated image segment depicting a road segment, wherein the rotated image segment is a rotated portion of an image of a road network, and wherein the rotated image segment is rotated to align the road segment with a target axis; applying the rotated image segment as input to a machine learning model to obtain a lane-marked rotated image segment; determining a degree of rotation with respect to the target axis of the rotated image segment; counter rotating the lane-marked rotated image segment by the degree of rotation to obtain a lane-marked image segment; and annotating the image of the road network based at least in part on the lane-marked image segment to obtain an annotated image.

18 . The non-transitory computer-readable storage medium of claim 17 , wherein the operations further comprise:
generating an overlay layer that includes lane markings from the lane-marked rotated image segment or the lane-marked image segment; and providing the overlay layer to one or more of: a vehicle routing system, a driver-assist system, or an autonomous vehicle system.

19 . The non-transitory computer-readable storage medium of claim 17 , wherein annotating the image of the road network comprises replacing a corresponding portion of the image with the lane-marked image segment.

20 . The non-transitory computer-readable storage medium of claim 17 , wherein annotating the image of the road network comprises applying lane markings from the lane-marked image segment to a corresponding portion of the image.

Assignees

• Didi Res America Llc

Inventors

• Hou Tingbo
• Zhang Yan

Classifications

• G06T3/60 Primary

  Rotation of whole images or parts thereof · CPC title

• G06V10/772

  Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries · CPC title

• G06V20/588

  Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road · CPC title

• G06F18/211

  Selection of the most significant subset of features · CPC title

• G06T11/23

  using straight lines or curves · CPC title