Virtual lane mark generation

Patent metadata

Field	Value
Publication number	US-2023360408-A1
Application number	US-202217738328-A
Country	US
Kind code	A1
Filing date	May 6, 2022
Priority date	May 6, 2022
Publication date	Nov 9, 2023
Grant date	—

Abstract

A vehicle, system method for operating the vehicle is disclosed. The system includes a camera and a processor. The camera is configured to obtain a camera image of a road segment. The processor determines a location of a road edge for the road segment within the camera image, obtains a lane attribute for the road segment, generates a virtual lane mark for the road segment based on the road edge and the lane attribute, and moves the vehicle along the road segment by tracking the virtual lane mark.

First claim

What is claimed is:

1 . A method for operating a vehicle, comprising:
obtaining a camera image of a road segment using a camera at the vehicle; determining a location of a road edge for the road segment within the camera image; obtaining a lane attribute for the road segment; generating a virtual lane mark for the road segment based on the road edge and the lane attribute; and moving the vehicle along the road segment by tracking the virtual lane mark.

2 . The method of claim 1 , further comprising fitting a lane model equation to pixels in the camera image indicative of the road edge and generating the virtual lane mark using the lane model equation and the lane attribute.

3 . The method of claim 2 , further comprising transforming the camera image from a pixel coordinate system to a bird's eye view coordinate system and fitting the virtual lane mark to the lane model equation in the bird's eye view coordinate system.

4 . The method of claim 1 , further comprising obtaining a road segmentation image from the camera image and inputting the road segmentation image and the lane attribute into a neural network to determine the virtual lane mark.

5 . The method of claim 4 , wherein the neural network is at least one of: (i) a multilayer perception network; (ii) an autoencoder/decoder network; (iii) a conditional generative adversarial network; and (iv) a convolutional neural network.

6 . The method of claim 1 , wherein the lane attribute includes at least one of: (i) a number of lanes in the road segment; (ii) a lane width for the road segment; (iii) a host lane index; (iv) a presence of an exit lane in the road segment; (v) a presence of a merging lane in the road segment; and (vi) a presence of an intersection in the road segment.

7 . The method of claim 6 , further comprising obtaining the lane attribute from at least one of: (i) a Global Positioning Satellite server; (ii) a map server; and (iii) a trajectory of one or more surrounding agents.

8 . A system for operating a vehicle, comprising:
a camera configured to obtain a camera image of a road segment; and a processor configured to:
determine a location of a road edge for the road segment within the camera image;
obtain a lane attribute for the road segment;
generate a virtual lane mark for the road segment based on the road edge and the lane attribute; and
move the vehicle along the road segment by tracking the virtual lane mark.

9 . The system of claim 8 , wherein the processor is further configured to fit a lane model equation to pixels in the camera image indicative of the road edge and generating the virtual lane mark using the lane model equation and the lane attribute.

10 . The system of claim 9 , wherein the processor is further configured to transform the camera image from a pixel coordinate system to a bird's eye view coordinate system and fit the virtual lane mark to the lane model equation in the bird's eye view coordinate system.

11 . The system of claim 8 , wherein the processor is further configured to operate a neural network to receive the lane attribute and a road segmentation image generated from the camera image and to output the virtual lane mark.

12 . The system of claim 11 , wherein the neural network is at least one of: (i) a multilayer perception network; (ii) an autoencoder/decoder network; (iii) a conditional generative adversarial network; and (iv) a convolutional neural network.

13 . The system of claim 8 , wherein the lane attribute includes at least one of: (i) a number of lanes in the road segment; (ii) a lane width for the road segment; (iii) a host lane index; (iv) a presence of an exit lane in the road segment; (v) a presence of a merging lane in the road segment; and (vi) a presence of an intersection in the road segment.

14 . The system of claim 13 , wherein the processor is further configured to obtain the lane attribute from at least one of: (i) a Global Positioning Satellite server; (ii) a map server; and (iii) a trajectory of one or more surrounding agents.

15 . A vehicle, comprising:
a camera configured to obtain a camera image of a road segment; and a processor configured to:
determine a location of a road edge for the road segment within the camera image;
obtain a lane attribute for the road segment;
generate a virtual lane mark for the road segment based on the road edge and the lane attribute; and
move the vehicle along the road segment by tracking the virtual lane mark.

16 . The vehicle of claim 15 , wherein the processor is further configured to fit a lane model equation to pixels in the camera image indicative of the road edge and generating the virtual lane mark using the lane model equation and the lane attribute.

17 . The vehicle of claim 16 , wherein the processor is further configured to transform the camera image from a pixel coordinate system to a bird's eye view coordinate system and fit the virtual lane mark to the lane model equation in the bird's eye view coordinate system.

18 . The vehicle of claim 15 , the processor is further configured to operate a neural network to receive the lane attribute and a road segmentation image generated from the camera image and to output the virtual lane mark.

19 . The vehicle of claim 18 , wherein the neural network is at least one of: (i) a multilayer perception network; (ii) an autoencoder/decoder network; (iii) a conditional generative adversarial network; and (iv) a convolutional neural network.

20 . The vehicle of claim 15 , wherein the lane attribute includes at least one of: (i) a number of lanes in the road segment; (ii) a lane width for the road segment; (iii) a host lane index; (iv) a presence of an exit lane in the road segment; (v) a presence of a merging lane in the road segment; and (vi) a presence of an intersection in the road segment.

Assignees

• Gm Global Tech Operations Llc

Inventors

• Jiang Shengbing
• Lin Wen-Chiao
• Yang Zhongyu
• Zhang Yilu

Classifications

• G06V20/588 Primary

  Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road · CPC title

• B60W30/12

  Lane keeping · CPC title

• B60W2420/42

  Operations & Transport · mapped topic

• B60W2552/53

  Road markings, e.g. lane marker or crosswalk · CPC title

• B60W2554/4045

  Intention, e.g. lane change or imminent movement · CPC title