Attention guidance for ground control labeling in street view imagery

Patent metadata

Field	Value
Publication number	US-2022058844-A1
Application number	US-202017110735-A
Country	US
Kind code	A1
Filing date	Dec 3, 2020
Priority date	Aug 18, 2020
Publication date	Feb 24, 2022
Grant date	—

Abstract

The disclosure provides methods, apparatus, and products for attention guidance and labeling. In one aspect, a method comprises receiving metadata associated with a perspective image captured by an image capture device located at the image position and having the image pose, wherein the metadata comprises the image position and the image pose. The method comprises defining a field-of-view indicator having a first indicator position located at the image position and having the image pose; receiving ground control point (GCP) information identifying a GCP, wherein the GCP information comprises a GCP position; defining a GCP indicator having a second indicator position located at the GCP position; and causing display of an indicator layer comprising the field-of-view indicator and the GCP indicator in a second portion of an interactive user interface (IUI) of a labeling tool, wherein at least the perspective image is displayed in a first portion of the IUI.

First claim

1 . A method comprising:
receiving, by a processor of a user apparatus, metadata associated with a perspective image captured by an image capture device located at an image position and having an image pose, wherein the metadata comprises the image position and the image pose; defining, by the user apparatus, a field-of-view indicator having a first indicator position located at the image position and having the image pose, wherein the field-of-view-indicator (a) is an interactive user interface (IUI) display element and (b) is configured to represent the field-of-view of the perspective image; receiving, by the processor of the user apparatus, ground control point (GCP) information identifying a GCP, wherein the GCP information comprises a GCP position based at least in part on a location of the GCP and wherein a GCP is a feature associated with a ground truth position; defining, by the user apparatus, a GCP indicator having a second indicator position located at the GCP position, wherein the GCP indicator (a) is another IUI display element and (b) is configured to represent a location of the GCP; generating, by the user apparatus, an indicator layer comprising the field-of-view indicator and the GCP indicator, the indicator layer configured to be rendered and displayed via output circuitry; and causing, by the user apparatus, display of the indicator layer comprising the field-of-view indicator and the GCP indicator in a second portion of an IUI of a labeling tool, wherein at least the perspective image is displayed in a first portion of the IUI of the labeling tool and the labeling tool is provided via output circuitry of the user apparatus.

2 . The method of claim 1 , wherein the indicator layer comprising the field-of-view indicator and the GCP indicator is displayed overlaid on a digital map layer in the second portion of the IUI of the labeling tool, the digital map layer being one of a satellite layer, a two-dimensional model layer, or a three-dimensional model layer.

3 . The method of claim 1 , wherein the field-of-view indicator comprises a configurable representation of an approximate field-of-view of the image capture device when the perspective image was captured.

4 . The method of claim 3 , wherein the field-of-view indicator comprises two lines extending from the first indicator position, the two lines defining a field-of-view angle that is oriented based at least in part on the image pose, the field-of-view angle representing the approximate field-of-view.

5 . The method of claim 3 , further comprising providing an alert via the IUI of the labeling tool based at least in part on the GCP indicator being located within the approximate field-of-view.

6 . The method of claim 1 , wherein the image position and the image pose are determined at least in part by a location sensor of a probe apparatus, the image capture device being coupled to the probe apparatus.

7 . The method of claim 6 , wherein the image position comprises a latitude, longitude, and elevation of the probe apparatus, and the image pose comprises an orientation of the probe apparatus.

8 . The method of claim 1 , further comprising:
receiving, by the processor of the user apparatus, metadata associated with a second perspective image, wherein the metadata associated with the second perspective image comprises a second image position and a second image pose; defining, by the user apparatus, a second image indicator having a third indicator position located at the second image position; and causing display of the second image indicator in the second portion of the IUI of the labeling tool.

9 . The method of claim 8 , further comprising:
defining, by the user apparatus, a second field-of-view indicator having the third indicator position located at the second image position and having the second image pose; defining, by the user apparatus, a first image indicator having the first indicator position located at the image position; and responsive to user interaction with input circuitry of the user apparatus, causing display of a second indicator layer comprising the second field-of-view indicator, the first image indicator, and the GCP indicator via the second portion of the IUI of the labeling tool.

10 . The method of claim 1 , wherein the GCP is an identifiable, static feature located within a predetermined distance from the image position.

11 . The method of claim 10 , wherein the GCP is at least partially visible in the perspective image.

12 . The method of claim 1 , wherein the GCP position is a ground truth position.

13 . The method of claim 1 , wherein the GCP information further comprises a plurality of control images, the GCP being visible in each of the plurality of control images.

14 . The method of claim 13 , further comprising causing display of the plurality of control images in the first portion of the IUI of the labeling tool.

15 . An apparatus comprising at least one processor and at least one memory storing computer program code, the at least one memory and the computer program code are configured to, with the processor, cause the apparatus to at least:
receive metadata associated with a perspective image captured by an image capture device located at the image position and having the image pose, wherein the metadata comprises the image position and the image pose; define a field-of-view indicator having a first indicator position located at the image position and having the image pose, wherein the field-of-view-indicator (a) is an interactive user interface (IUI) display element and (b) is configured to represent the field-of-view of the perspective image; receive ground control point (GCP) information identifying a GCP, wherein the GCP information comprises a GCP position based at least in part on a location of the GCP and wherein a GCP is a feature associated with a ground truth position; define a GCP indicator having a second indicator position located at the GCP position, wherein the GCP indicator (a) is another IUI display element and (b) is configured to represent a location of the GCP; generate an indicator layer comprising the field-of-view indicator and the GCP indicator, the indicator layer configured to be rendered and displayed via output circuitry; and cause display of the indicator layer comprising the field-of-view indicator and the GCP indicator in a second portion of an IUI of a labeling tool, wherein at least the perspective image is displayed in a first portion of the IUI of the labeling tool and the labeling tool is provided via output circuitry.

16 . The apparatus of claim 15 , wherein the indicator layer comprising the field-of-view indicator and the GCP indicator is displayed overlaid on a digital map layer in the second portion of the IUI of the labeling tool, the digital map layer being one of a satellite layer, a two-dimensional model layer, or a three-dimensional model layer.

17 . The apparatus of claim 15 , wherein the field-of-view indicator comprises a configurable representation of an approximate field-of-view of the image capture device when the perspective image was captured.

18 . The apparatus of claim 15 , further caused to at least:
receive metadata associated with a second perspective image, wherein the metadata associated with the second perspective image comprises a second image position and a second image pose; and define a second image indicator having a third indicator position located at the second image position; and cause display of the second image indicator in the second portion of the

Assignees

• Here Global Bv

Inventors

• Chen Zhanwei
• Pojman Nick
• Lawlor David

Classifications

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• G06V20/182

  Network patterns, e.g. roads or rivers · CPC title

• G06T11/00

  Two-dimensional [2D] image generation · CPC title

• G06F3/0484

  for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range · CPC title

• G09G2380/10

  Automotive applications · CPC title