Method and system for generating navigation data for a geographical location

Patent metadata

Field	Value
Publication number	US-11656090-B2
Application number	US-201916425587-A
Country	US
Kind code	B2
Filing date	May 29, 2019
Priority date	Oct 8, 2018
Publication date	May 23, 2023
Grant date	May 23, 2023

Abstract

An approach is provided for generating navigation data of a geographical location. The approach involves identifying a landmark located along a source road from a source image and segmenting the source image using a deep learning model to identify a segmentation mask. The approach also involves generating a template image based on the segmentation mask and a street image of the landmark, and matching the template image successively with a sequence of images of the landmark to determine a confidence score. The approach further involves, identifying a first image from the sequence of images with confidence score below a predetermined threshold, and selecting a second image with confidence score above the predetermined threshold from the sequence of images. The approach further involves calculating a visibility distance of the landmark based on the source image and the second image, and generating the navigation data based on the calculated visibility distance.

First claim

What is claimed is:

1. A computer-implemented method for generating navigation data of a geographical location, the method comprising:
identifying at least one landmark located along a source road in the geographical location from a source image captured by an imaging device;
segmenting the source image using a deep learning model to identify a segmentation mask;
generating a template image based on the segmentation mask and a street image of the at least one landmark, wherein the street image comprises a 2D footprint of the at least one landmark falling within a field-of-view of the imaging device;
matching the template image successively with a sequence of images of the at least one landmark to determine a confidence score for each image in the sequence of images;
identifying a first image from the sequence of images whose confidence score is below a predetermined threshold;
selecting a second image from the sequence of images, that is immediately preceding the first image in the sequence of images, wherein the confidence score of the second image is above the predetermined threshold;
calculating a visibility distance of the at least one landmark based on the source image and the second image;
generating the navigation data based on the calculated visibility distance;
determining a shape figure of the at least one landmark in the template image;
determining a largest fitting rectangle fitting inside the shape figure;
matching the template image successively with the sequence of images of the at least one landmark based on the largest fitting rectangle.

2. The method of claim 1 , wherein the largest fitting rectangle is a vertically fitting rectangle bounding maximum height of the shape figure.

3. The method of claim 2 , wherein the sequence of images is captured by the imaging device from multiple points of view on the source road, wherein each of the sequence of images has at least a portion of the at least one landmark, and wherein a size of the at least one landmark corresponding to the at least portion is different in each of the sequence of images, and wherein the largest fitting rectangle is resized on the basis of the size of the at least one landmark corresponding to the at least portion in each of the sequence of images.

4. The method of claim 1 , wherein the source image is segmented for detection and localization of the at least one landmark.

5. The method of claim 1 further comprising:
detecting one or more potential landmarks located along the source road in the geographical location; and
selecting the at least one landmark, for determining the visibility distance, having better visibility in the sequence of images among the one or more potential landmarks.

6. The method of claim 1 further comprising:
determining one or more navigation routes from the source road to each of one or more connecting roads using the at least one landmark as a reference point and the visibility distance as a reference distance based on the generated navigation data;
storing the determined one or more navigation routes; and
performing a voice over process for providing turn-by-turn navigation instructions for the stored one or more navigation routes.

7. The method of claim 6 further comprising:
recognizing one or more physical features of the at least one landmark from at least one of the sequence of images; and
using the one or more recognized physical features of the at least one landmark as one or more references in the one or more navigation routes.

8. The method of claim 7 , wherein a discernible color of the at least one landmark is recognized from at least one of the sequence of images utilizing prominent pixel intensities techniques.

9. A system for generating navigation data of a geographical location, the system comprising:
at least one database configured to store a sequence of images of at least one landmark located along a source road in the geographical location; and
a computing arrangement configured to:
identify the at least one landmark from a source image captured by an imaging device;
segment the source image using a deep learning model to identify a segmentation mask;
generate a template image based on the segmentation mask and a street image of the at least one landmark, wherein the street image comprises a 2D footprint of the at least one landmark falling within a field-of-view of the imaging device;
match the template image successively with the sequence of images to determine a confidence score for each image in the sequence of images;
identify a first image from the sequence of images whose confidence score is below a predetermined threshold;
select a second image from the sequence of images, that is immediately preceding the first image in the sequence of images, wherein the confidence score of the second image is above the predetermined threshold;
calculate a visibility distance of the at least one landmark based on the source image and the second image;
generate the navigation data based on the calculated visibility distance;
determine a shape figure of the at least one landmark in the template image;
determine a largest fitting rectangle fitting inside the shape figure; and
match the template image successively with the sequence of images based on the largest fitting rectangle.

10. The system of claim 9 , wherein the largest fitting rectangle is a vertically fitting rectangle bounding maximum height of the shape figure.

11. The system of claim 10 , wherein the imaging device is configured to capture the sequence of images from multiple points of view on the source road, wherein each of the sequence of images has at least a portion of the at least one landmark, and wherein a size of the at least one landmark corresponding to the at least portion is different in each of the sequence of images, and wherein the largest fitting rectangle is resized on the basis of the size of the at least one landmark corresponding to the at least portion in each of the sequence of images.

12. The system of claim 9 , wherein the source image is segmented for detection and localization of the at least one landmark.

13. The system of claim 9 , wherein the computing arrangement is further configured to:
detect one or more potential landmarks located along the source road in the geographical location; and
select the at least one landmark, to determine the visibility distance, having better visibility in the sequence of images among the one or more potential landmarks.

14. The system of claim 9 , wherein the computing arrangement is further configured to:
determine one or more navigation routes from the source road to each of one or more connecting roads using the at least one landmark as a reference point and the visibility distance as a reference distance based on the generated navigation data;
store the determined one or more navigation routes; and
perform a voice over process to provide turn-by-turn navigation instructions for the stored one or more navigation routes.

15. The system of claim 14 , wherein the computing arrangement is further configured to:
recognize one or more physical features of the at least one landmark from at least one of the sequence of images; and
use the one or more recognized physical features of the at least one landmark as one or more references in the one or more navigation routes.

16. The system of claim 15 , wherein a discernible color of the at least one landmark is recognized from at least one of the sequence of images utilizing prominent pixel intensities techniques.

17. A computer program

Assignees

• Here Global Bv

Inventors

• Soni Abhilshit
• Pachaar Akshay

Classifications

• G01C21/3602

  Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera · CPC title

• G08G1/096861

  where the immediate route instructions are output to the driver, e.g. arrow signs for next turn · CPC title

• G01C21/3644 Primary

  Landmark guidance, e.g. using POIs or conspicuous other objects · CPC title

• G06T7/11

  Region-based segmentation · CPC title

• G06T7/74

  involving reference images or patches · CPC title