Systems and methods for aligning map data

Patent metadata

Field	Value
Publication number	US-11036986-B2
Application number	US-201916271526-A
Country	US
Kind code	B2
Filing date	Feb 8, 2019
Priority date	Feb 8, 2019
Publication date	Jun 15, 2021
Grant date	Jun 15, 2021

Abstract

Systems, methods, and non-transitory computer-readable media can receive a geometric map and a semantic map associated with a geographic area, the semantic map comprising semantic data associated with vehicle navigation. A first semantic position estimate associated with a first piece of semantic data contained in the semantic map is generated based on semantic data location information associated with the first piece of semantic data. A final position for the first semantic position estimate is received. One or more three-dimensional semantic labels are applied to the geometric map based on the final position of the first semantic position estimate. A warped semantic map is generated. Generating the warped semantic map comprises warping the semantic map based on the one or more three-dimensional semantic labels.

First claim

What is claimed is:

1. A computer-implemented method comprising:
receiving, by a computing system, a geometric map and a semantic map associated with a geographic area, the semantic map including semantic data associated with vehicle navigation;
generating, by the computing system, a semantic position estimate based on a portion of the semantic data associated with an object, wherein the portion of the semantic data is included in the semantic map;
receiving, by the computing system, an updated position of the semantic position estimate based on aligning the semantic position estimate with one or more corresponding physical features associated with the object in the geometric map;
applying, by the computing system, one or more three-dimensional semantic labels to the geometric map based on the updated position of the semantic position estimate; and
generating, by the computing system, a warped semantic map, wherein the generating the warped semantic map is based on the one or more three-dimensional semantic labels.

2. The computer-implemented method of claim 1 , further comprising generating one or more two-dimensional semantic labels that indicate a position of the portion of the semantic data in an image associated with the geometric map based on the updated position of the semantic position estimate.

3. The computer-implemented method of claim 1 , wherein the semantic position estimate is applied to a first image of a set of images associated with the geometric map, and subsequently, used to estimates a position of the portion of the semantic data within the first image.

4. The computer-implemented method of claim 3 , wherein the receiving the updated position of the semantic position estimate comprises:
receiving an updated two-dimensional position of the semantic position estimate of the first image.

5. The computer-implemented method of claim 4 , wherein the receiving the updated position of the semantic position estimate comprises:
receiving a user input adjusting a position of the semantic position estimate of the first image.

6. The computer-implemented method of claim 4 , further comprising:
applying one or more three-dimensional semantic labels to the geometric map based on the updated position of the semantic position estimate, wherein the applying the one or more three-dimensional semantic labels comprises determining a region where a view ray associated with the first image intersects a surface in the geometric map, wherein the one or more three-dimensional semantic labels are applied to the geometric map based on the region where the view ray intersects the surface.

7. The computer-implemented method of claim 4 , wherein the applying one or more three-dimensional semantic labels to the geometric map based on the updated position of the semantic position estimate comprises translating the updated two-dimensional position of the semantic position estimate of the first image into a three-dimensional position within the geometric map.

8. The computer-implemented method of claim 7 , wherein the updated two-dimensional position of the semantic position estimate within the first image is translated into a three-dimensional position within the geometric map based on image location information associated with the first image and camera information associated with the first image, wherein the camera information comprises camera position, orientation, and direction information for a camera when the first image was captured.

9. The computer-implemented method of claim 1 , wherein the generating the warped semantic map is performed as an optimization problem that warps the semantic map based on a set of constraints.

10. The computer-implemented method of claim 1 , wherein the semantic data includes at least one of a position of a lane marker, an orientation of the lane marker, a position of a boundary, an orientations of the boundary, a lane direction, a speed limit, or a position of a road feature.

11. A system comprising:
at least one processor; and
a memory storing instructions that, when executed by the at least one processor, cause the system to perform:
receiving a geometric map and a semantic map associated with a geographic area, the semantic map including semantic data associated with vehicle navigation;
generating a semantic position estimate based on a portion of the semantic data associated with an object, wherein the portion of the asemantic data is included in the semantic map;
receiving an updated position of the semantic position estimate based on aligning the semantic position estimate with one or more corresponding physical features associated with the object in the geometric map;
applying one or more three-dimensional semantic labels to the geometric map based on the updated position of the semantic position estimate; and
generating a warped semantic map, wherein the generating the warped semantic map is based on the one or more three-dimensional semantic labels.

12. The system of claim 11 , wherein the instructions, when executed by the at least one processor, further cause the system to perform:
generating one or more two-dimensional semantic labels that indicate a position of the portion of the semantic data in an image associated with the geometric map based on the updated position of the semantic position estimate.

13. The system of claim 11 , wherein the semantic position estimate is applied to a first image of a set of images associated with the geometric map, and subsequently, used to estimates a position of the portion of the semantic data within the first image.

14. The system of claim 13 , wherein the receiving the updated position of the semantic position estimate comprises:
receiving a final two-dimensional position of the semantic position estimate of the first image.

15. The system of claim 14 , wherein the receiving the updated position of the semantic position estimate comprises:
receiving a user input adjusting a position of the semantic position estimate of the first image.

16. A non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of a computing system, cause the computing system to perform:
receiving a geometric map and a semantic map associated with a geographic area, the semantic map including semantic data associated with vehicle navigation;
generating a semantic position estimate based on a portion of the semantic data associated with an object, wherein the portion of the semantic data is included in the semantic map;
receiving an updated position of the semantic position estimate based on aligning the semantic position estimate with one or more corresponding physical features associated with the object in the geometric map;
applying one or more three-dimensional semantic labels to the geometric map based on the updated position of the semantic position estimate; and
generating a warped semantic map, wherein the generating the warped semantic map is based on the one or more three-dimensional semantic labels.

17. The non-transitory computer-readable storage medium of claim 16 , wherein the instructions, when executed by the at least one processor of the computing system, further cause the computing system to perform:
generating one or more two-dimensional semantic labels that indicate a position of the portion of the semantic data in an image associated with the geometric map based on the updated position of the semantic position estimate.

18. The non-transitory computer-readable storage medium of claim 16 , wherein the s

Assignees

• Lyft Inc

Inventors

• Dubroy Patrick
• Hess Wolfgang
• Marschner Clemens
• Praun Emil
• Rapp Holger
• Schiwietz Thomas
• Shet Vinay

Classifications

• G06V20/20 Primary

  in augmented reality scenes · CPC title

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• G06T2207/10024

  Color image · CPC title

• G06T7/74

  involving reference images or patches · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title