Location determination using street view images

Patent metadata

Field	Value
Publication number	US-10628959-B2
Application number	US-201715585721-A
Country	US
Kind code	B2
Filing date	May 3, 2017
Priority date	May 3, 2017
Publication date	Apr 21, 2020
Grant date	Apr 21, 2020

Abstract

A method, a computer program product, and a computer system for determining a location using street view images. A mobile device obtains an image captured by the mobile device, obtains a set of street view images, and obtains a first graph of the image captured by the mobile device and a plurality of second graphs of the set of the street view images. The first graph includes nodes representing interest points in the image captured by the mobile device, and the plurality of the second graphs includes nodes representing interest points in the set of the street view images. The mobile device determines a location of the mobile device, based on relevance between the first graph and the plurality of the second graphs.

First claim

What is claimed is:

1. A method comprising:
obtaining, by a mobile device, an image captured by the mobile device;
obtaining, by the mobile device, a set of street view images;
obtaining, by the mobile device, a first graph of the image captured by the mobile device and a plurality of second graphs of the set of the street view images, the first graph including nodes representing interest points in the image captured by the mobile device, and the plurality of the second graphs including nodes representing interest points in the set of the street view images;
determining, by the mobile device, a location of the mobile device, based on relevance between the first graph and the plurality of the second graphs;
generating, by the mobile device, a sub-graph from one of the plurality of the second graphs;
rotating, by the mobile device, the sub-graph by a plurality of angles to generate a plurality of sub-graphs;
determining, by the mobile device, relevance between the first graph and the plurality of the sub-graphs; and
determining, by the mobile device, the location of the mobile device, based on the relevance between the first graph and the plurality of the sub-graphs.

2. The method of claim 1 , wherein the obtaining the first graph of the image captured by the mobile device and the plurality of the second graphs of the set of the street view images comprises:
identifying, by the mobile device, a plurality of interest points in the image captured by the mobile device and the set of the street view images;
determining, by the mobile device, a plurality of edges between a plurality of nodes representing the plurality of the interest points; and
generating, by the mobile device, the first graph and the plurality of the second graphs, based on the plurality of nodes and the plurality of edges.

3. The method of claim 2 , wherein the determining the plurality of the edges between the plurality of the nodes representing the plurality of interest points comprises:
determining, by the mobile device, a horizontal value, based on a horizontal position relationship between two nodes;
determining, by the mobile device, a vertical value based on a vertical position relationship between the two nodes; and
determining, by the mobile device, an edge value representing a position relationship between the two nodes, based on the horizontal value and the vertical value.

4. The method of claim 2 , wherein the determining the plurality of the edges between the plurality of the nodes representing the plurality of the interest points comprises:
determining, by the mobile device, a first set of edge values, each of which representing a position relationship between two of the plurality of the nodes in the first graph; and
determining, by the mobile device, a second set of edge values, each of which representing a position relationship between two of the plurality of the nodes in one of the plurality of the second graphs.

5. The method of claim 4 , wherein the determining the location of the mobile device comprises:
determining, by the mobile device, a match degree between the first set of the edge values and the second set of the edge values; and
in response to determining that the match degree exceeds a predefined threshold, obtaining, by the mobile device, a location of the one of the plurality of the second graphs, and determining, by the mobile device, the location of the one of the plurality of the second graphs as the location of the mobile device.

6. The method of claim 5 , wherein the determining the match degree between the first set of the edge values and the second set of the edge values comprises:
dividing, by the mobile device, nodes in the first graph and the plurality of second graphs into a plurality of groups according to an image feature of each node; and
determining, by the mobile device, a match degree between a first edge value in the first set of the edge values and a second edge value in the second set of the edge values, first two nodes associated with the first edge value and second two nodes associated with the second edge value belonging to a same group of the plurality of groups.

7. The method of claim 1 , further comprising:
displaying, by the mobile device, the location of the mobile device on a map application; and
providing, by the mobile device, a navigational recommendation to a destination on the map application.

8. The method of claim 1 , the obtaining the set of the street view images comprises:
determining, by the mobile device, geographic information of the mobile device; and
obtaining, by the mobile device, the set of the street view images based on the geographic information.

9. A computer system comprising:
a processing unit;
a memory coupled to the processing unit and storing instructions thereon, the instructions, when executed by the processing unit, performing acts including:
obtaining, by a mobile device, an image captured by the mobile device;
obtaining, by the mobile device, a set of street view images;
obtaining, by the mobile device, a first graph of the image captured by the mobile device and a plurality of second graphs of the set of the street view images, the first graph including nodes representing interest points in the image captured by the mobile device, and the plurality of the second graphs including nodes representing interest points in the set of the street view images;
determining, by the mobile device, a location of the mobile device, based on relevance between the first graph and the plurality of the second graphs;
generating, by the mobile device, a sub-graph from one of the plurality of the second graphs;
rotating, by the mobile device, the sub-graph by a plurality of angles to generate a plurality of sub-graphs;
determining, by the mobile device, relevance between the first graph and the plurality of the sub-graphs; and
determining, by the mobile device, the location of the mobile device, based on the relevance between the first graph and the plurality of the sub-graphs.

10. The computer system of claim 9 , wherein the obtaining the first graph of the image captured by the mobile device and the plurality of the second graphs of the set of the street view images comprises:
identifying, by the mobile device, a plurality of interest points in the image captured by the mobile device and the set of the street view images;
determining, by the mobile device, a plurality of edges between a plurality of nodes representing the plurality of the interest points; and
generating, by the mobile device, the first graph and the plurality of the second graphs, based on the plurality of nodes and the plurality of edges.

11. The computer system of claim 10 , wherein the determining the plurality of the edges between the plurality of the nodes representing the plurality of interest points comprises:
determining, by the mobile device, a horizontal value, based on a horizontal position relationship between two nodes;
determining, by the mobile device, a vertical value based on a vertical position relationship between the two nodes; and
determining, by the mobile device, an edge value representing a position relationship between the two nodes, based on the horizontal value and the vertical value.

12. The computer system of claim 10 , wherein the determining the plurality of the edges between the plurality of the nodes representing the plurality of the interest points comprises:
determining, by the mobile device, a first set of edge values, each of which representing a position relationship between two of the plurality of the nodes in the first graph; and
determining, by the mobile device, a second se

Assignees

• IBM

Inventors

• Chen xiao long
• Lv Na
• Yan Zhe
• Zhai Yun
• Zhao Zhuo

Classifications

• G06T2207/30244

  Camera pose · CPC title

• G06T7/74 Primary

  involving reference images or patches · CPC title

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title

• H04W88/02

  Terminal devices · CPC title

• G06T2207/20072

  Graph-based image processing · CPC title