Manual curation tool for map data using aggregated overhead views

Patent metadata

Field	Value
Publication number	US-11488353-B2
Application number	US-202016836805-A
Country	US
Kind code	B2
Filing date	Mar 31, 2020
Priority date	Mar 31, 2020
Publication date	Nov 1, 2022
Grant date	Nov 1, 2022

Abstract

Examples disclosed herein may involve (i) obtaining a first layer of map data associated with sensor data capturing a geographical area, the first layer of map data comprising an aggregated overhead-view image of the geographical area, where the aggregated overhead-view image is generated from aggregated pixel values from a plurality of images associated with the geographical area, (ii) obtaining a second layer of map data, the second layer of map data comprising label data for the geographical area derived from the aggregated overhead-view image of the geographical area, and (iii) causing the first layer of map data and the second layer of map data to be presented to a user for curation of the label data.

First claim

What is claimed is:

1. A method comprising;
obtaining a first layer of map data associated with sensor data capturing a geographical area, the first layer of map data comprising an aggregated overhead-view image of the geographical area,
wherein obtaining the aggregated overhead-view image includes generating an aggregated overhead-view map from aggregated pixel values from a plurality of images that are ground-level images from a sensor with a limited field-of-view associated with the geographical area, wherein generating the aggregated overhead-view image of the geographical area includes using a ground map of the geographical area and the plurality of images of the geographical area,
wherein generating the first layer includes sampling points from relevant images of the plurality of images for respective segments of the aggregated overhead-view map according to intersections of rays extrapolated from the relevant images;
obtaining a second layer of map data, the second layer of map data comprising label data for the geographical area derived from the aggregated overhead-view image of the geographical area, including vehicle trajectory lines associated with vehicles that acquired the plurality of images, and points along the trajectory lines where the plurality of images were captured;
causing the first layer of map data and the second layer of map data to be presented to a user for curation of the label data, wherein the second layer provides cues within a display about the relevant images from the plurality of images used to generate the map data along with editable points for modifying the map data of the second layer.

2. The method as recited in claim 1 ,
wherein generating the first layer includes querying the plurality of images to identify the relevant images in which pixels of a specific segment of the aggregated overhead-view image occur by backpropagation of the rays for different cameras of the plurality of images to the specific segment.

3. The method as recited in claim 1 , wherein causing the first layer and the second layer to be presented includes highlighting defects between the first layer and the second layer according to an automated comparison of the first layer and the second layer, and presenting suggestions for adapting the second layer according to the defects.

4. The method as recited in claim 1 , wherein causing the first layer and the second layer to be presented includes acquiring electronic inputs to adjust at least the second layer, the electronic inputs include inputs for curation of the label data that comprises inputs from a group including verifying the label data, editing the label data, adding new label data to the label data, or removing incorrect or irrelevant label data from the label data based on a correspondence of the aggregated overhead-view image to the label data.

5. The method as recited in claim 1 , wherein the second layer of the map data is generated by automatically extracting label data from the aggregated overhead-view image of the geographical area, wherein the label data comprises semantic map data.

6. The method as recited in claim 1 , wherein the label data is automatically generated using one or more of: machine learning models; classifiers; or Generative Adversarial Networks.

7. The method as recited in claim 1 , wherein the label data comprises one or more of: lane boundaries; lane connectivity; speed limits; types of traffic elements; crosswalks; speed bumps; pedestrian paths or sidewalks; manhole covers; or curbs.

8. A method comprising:
receiving labeled map data of a geographical area from a mapping system, the labeled map data including:
a first layer of map data comprising an aggregated overhead-view image of a geographical area, wherein receiving the aggregated overhead-view image includes generating an aggregated overhead-view map from aggregated pixel values from a plurality of images that are ground-level images from a sensor with a limited field-of-view of the geographical area, wherein generating the aggregated overhead-view image of the geographical area includes using a ground map of the geographical area and the plurality of images of the geographical area,
wherein generating the first layer includes sampling points from relevant images of the plurality of images for respective segments of the aggregated overhead-view map according to intersections of rays extrapolated from the relevant images; and
a second layer of map data comprising label data for the geographical area derived from the aggregated overhead-view image of the geographical area, including vehicle trajectory lines associated with vehicles that acquired the plurality of images, and points along the trajectory lines where the plurality of images were captured, wherein the second layer provides cues within a display about the relevant images from the plurality of images used to generate the map data along with editable points for modifying the map data of the second layer;
displaying the first layer of map data and the second layer of map data to a user;
receiving user input comprising one or more adjustments to the label data; and
causing the label data to be adjusted in accordance with the user input.

9. The method as recited in claim 8 , wherein causing the label data to be adjusted in accordance with the user input comprises one of (i) adjusting the label data locally and then providing the adjusted label data to the mapping system or (ii) providing the user input to the mapping system and thereby causing mapping system to adjust the label data.

10. The method as recited in claim 8 , wherein causing the label data to be adjusted in accordance with the user input comprises causing the second layer of map data to be updated.

11. The method as recited in claim 8 , further comprising:
updating the second layer of the map in accordance with the user input.

12. The method as recited in claim 8 , wherein the one or more adjustments to the label data are based on one or more of: a set of guidelines; a set of instructions; one or more plug-ins for adjustment; or one or more tools for adjustment input.

13. The method as recited in claim 8 , wherein the one or more adjustments of the label data comprise one or more of: visual manipulation; determining abnormalities; determining alignments/misalignments; inputting one or more annotations; selecting/de-selecting one or more of the label data; removing/re-embedding one or more of the label data; hiding/exposing one or more of the label data; or enlargement/diminution of one or more of the label data.

14. The method as recited in claim 8 , wherein the the first layer and the second layer are stored in one or more local system or a remote system.

15. The method as recited in claim 8 , further comprising:
causing a global map to be updated in accordance with the user input.

16. The method as recited in claim 8 , wherein displaying the first layer of map data and the second layer of map data comprises displaying the second layer of the map overlaid on the first layer of the map.

17. A computer system comprising:
at least one processor;
at least one non-transitory computer-readable medium;
program instructions stored on the at least one non-transitory computer-readable medium that are executable by the at least one processor such that the computer system is capable of:
receiving labeled map data of a geographical area from a mapping system, the labeled map data including:
a first layer of map data comprising an aggregated overhead-view image of a geographical area, wherein receiving the agg

Assignees

• Woven Planet North America Inc

Inventors

• Dubroy Patrick
• Marschner Clemens
• Richert Wilhelm
• Schiwietz Thomas

Classifications

• G06V20/13

  Satellite images · CPC title

• G06T17/05 Primary

  Geographic models · CPC title

• G06V30/19167

  Active pattern learning · CPC title

• G06V10/26

  Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion · CPC title

• G06T11/60 Primary

  Creating or editing images; Combining images with text · CPC title