Electronic device localization based on imagery

Patent metadata

Field	Value
Publication number	US-11195049-B2
Application number	US-201815923492-A
Country	US
Kind code	B2
Filing date	Mar 16, 2018
Priority date	Apr 25, 2014
Publication date	Dec 7, 2021
Grant date	Dec 7, 2021

Abstract

An electronic device includes one or more imaging cameras. After a reset of the device or other specified event, the electronic device identifies an estimate of the device's pose based on location data such as Global Positioning System (GPS) data, cellular tower triangulation data, wireless network address location data, and the like. The one or more imaging cameras may be used to capture imagery of the local environment of the electronic device, and this imagery is used to refine the estimated pose to identify a refined pose of the electronic device. The refined pose may be used to identify additional imagery information, such as environmental features, that can be used to enhance the location based functionality of the electronic device.

First claim

What is claimed is:

1. A method, comprising,
in response to a reset event at an electronic device, identifying at the electronic device an origin in a first frame of reference corresponding to a free frame of reference;
calculating an estimated first pose of the electronic device in a second frame of reference corresponding to a geographic frame of reference, the first pose corresponding to the origin;
identifying a refined first pose with respect to the geographic frame of reference by refining the estimated first pose based on a first image captured by an imaging camera at the electronic device;
concurrent with calculating the estimated first pose and identifying the refined first pose, tracking at least one change in position of the electronic device in the first frame of reference relative to the origin; and
translating the refined first pose of the electronic device to a second pose in the second frame of reference based on the change in position of the electronic device in the first frame of reference.

2. The method of claim 1 , wherein calculating the estimated first pose comprises calculating the estimated first pose based on Global Positioning System data capture at the electronic device.

3. The method of claim 1 , wherein calculating the estimated first pose comprises calculating the estimated first pose based on wireless tower triangulation information for the electronic device.

4. The method of claim 1 , wherein identifying the refined first pose comprises:
identifying a feature of the first image;
matching the feature to a first feature descriptor of a database of feature descriptors; and
identifying the refined first pose based on a pose associated with the first feature descriptor.

5. The method of claim 4 , wherein identifying the refined first pose comprises:
identifying a plurality of feature descriptors based on the estimated first pose; and
searching the identified plurality of feature descriptors for the first feature descriptor.

6. The method of claim 4 , further comprising:
identifying the first feature descriptor based on a gravity direction of the first image identified at the electronic device.

7. The method of claim 4 , further comprising:
identifying a second feature descriptor based on the second pose; and
providing information based on the second feature descriptor to an application executing at the electronic device.

8. An electronic device, comprising,
an imaging camera; and
a processor to:
in response to a reset event at the electronic device, identify at the electronic device an origin in a first frame of reference corresponding to a free frame of reference;
calculating an estimated first pose of the electronic device in a second frame of reference corresponding to a geographic frame of reference, the first pose corresponding to the origin;
identifying a refined first pose with respect to the geographic frame of reference by refining the estimated first pose based on a first image captured by an imaging camera at the electronic device;
concurrent with calculating the estimated first pose, track at least one change in position of the electronic device in the first frame of reference relative to the origin; and
translate the refined first pose of the electronic device to a second pose in the second frame of reference based on the change in position in the first frame of reference.

9. The electronic device of claim 8 , wherein the estimated first pose is calculated based on Global Positioning System data capture at the electronic device.

10. The electronic device of claim 8 , wherein the estimated first pose is calculated based on wireless tower triangulation information for the electronic device.

11. The electronic device of claim 8 , wherein the processor is to identify the refined first pose by:
identifying a feature of the first image;
matching the feature to a first feature descriptor of a database of feature descriptors; and
identifying the refined first pose based on a pose associated with the first feature descriptor.

12. The electronic device of claim 11 , wherein the processor is to identify the refined first pose by:
identifying a plurality of feature descriptors based on the estimated first pose; and
searching the identified plurality of feature descriptors for the first feature descriptor.

13. The electronic device of claim 11 , wherein the processor is further to:
identify the first feature descriptor based on a gravity direction of the first image identified at the electronic device.

14. The electronic device of claim 8 , wherein the processor is further to:
identify a second feature descriptor based on the second pose; and
provide information based on the second feature descriptor to an application executing at the electronic device.

Assignees

• Google Llc

Inventors

• Hesch Joel
• Nerurkar Esha
• Mihelich Patrick

Classifications

• G06F18/253

  of extracted features · CPC title

• G01S19/485 Primary

  whereby the further system is an optical system or imaging system · CPC title

• G06V2201/07

  Target detection · CPC title

• G06T2207/30244

  Camera pose · CPC title

• G06K9/6232 Primary

  Physics · mapped topic