Automatic Detection of Noteworthy Locations

Patent metadata

Field	Value
Publication number	US-2019147303-A1
Application number	US-201816040262-A
Country	US
Kind code	A1
Filing date	Jul 19, 2018
Priority date	May 10, 2012
Publication date	May 16, 2019
Grant date	—

Abstract

By providing 3D representations of noteworthy locations for comparison with images, the 3D location of the imaging device, as well as the orientation of the device may be determined. The 3D location and orientation of the imaging device then allows for enhanced navigation in a collection of images, as well as enhanced visualization and editing capabilities. The 3D representations of noteworthy locations may be provided in a database that may be stored local or remote to the imaging device or a programmable device processing images obtained from the imaging device.

First claim

1 . A method for viewing images by a device, comprising:
capturing a first image, the first image associated with location data corresponding to a location from which the first image was captured; receiving a request to display the first image; identifying a predefined three-dimensional representation based on the location data; detecting one or more features in the first image; matching the detected one or more features in the first image with one or more features of the identified predefined three-dimensional representation; determining a three-dimensional location from which the first image was captured based, at least in part, on the matched one or more features and the location data; compositing the first image with a second image, the second image from a collection of images and associated with the detected one or more features in the first image; and presenting the composited image.

2 . The method of claim 1 , wherein the location data is obtained by obtaining geodata for the first image.

3 . The method of claim 2 , wherein obtaining geodata for the first image comprises obtaining GPS data from metadata of the first image.

4 . The method of claim 1 , wherein the predefined three-dimensional representation comprises a three dimensional model and wherein the second image is not captured by the device.

5 . The method of claim 1 , further comprising calculating an orientation of the first image based, at least in part, on the matching, wherein the orientation corresponds to a direction from which the first image was captured.

6 . The method of claim 1 , further comprising adding tags based on the one or more features of the identified predefined three-dimensional representation, and wherein the composited image includes the added tags.

7 . The method of claim 1 , further comprising presenting one or more directional indicators, the one or more directional indicators indicating navigation directions based on other images in the collection of images.

8 . A non-transitory program storage device comprising instructions stored thereon to cause one or more processors of a device to:
capture a first image, the first image associated with location data corresponding to a location from which the first image was captured; receive a request to display the first image; identify a predefined three-dimensional representation based on the location data; detect one or more features in the first image; match the detected one or more features in the first image with one or more features of the identified predefined three-dimensional representation; determine a three-dimensional location from which the first image was captured based, at least in part, on the matched one or more features and the location data; composite the first image with a second image, the second image from a collection of images and associated with the detected one or more features in the first image; and present the composited image.

9 . The non-transitory program storage device of claim 8 , wherein the location data is obtained by obtaining geodata for the first image.

10 . The non-transitory program storage device of claim 9 , wherein obtaining geodata for the first image comprises obtaining GPS data from metadata of the first image.

11 . The non-transitory program storage device of claim 8 , wherein the predefined three-dimensional representation comprises a three dimensional model and wherein the second image is not captured by the device.

12 . The non-transitory program storage device of claim 8 , wherein the instructions further cause the one or more processors to calculate an orientation of the first image based, at least in part, on the matching, wherein the orientation corresponds to a direction from which the first image was captured.

13 . The non-transitory program storage device of claim 8 , wherein the instructions further cause the one or more processors to add tags based on the one or more features of the identified predefined three-dimensional representation, and wherein the presented composited image includes the added tags.

14 . The non-transitory program storage device of claim 8 , wherein the instructions further cause the one or more processors to present one or more directional indicators, the one or more directional indicators indicating navigation directions based on other images in the collection of images.

15 . An electronic device, comprising:
a memory; one or more image capture devices; a display; a user interface; and one or more processors operatively coupled to the memory, wherein the one or more processors are configured to execute instructions causing the one or more processors to:
capture a first image, the first image associated with location data corresponding to a location from which the first image was captured;
receive a request to display the first image;
identify a predefined three-dimensional representation based on the location data;
detect one or more features in the first image;
match the detected one or more features in the first image with one or more features of the identified predefined three-dimensional representation;
determine a three-dimensional location from which the first image was captured based, at least in part, on the matched one or more features and the location data;
composite the first image with a second image, the second image from a collection of images and associated with the detected one or more features in the first image; and
present the composited image.

16 . The electronic device of claim 15 , wherein the location data is obtained by obtaining geodata for the first image.

17 . The electronic device of claim 16 , wherein obtaining geodata for the first image comprises obtaining GPS data from metadata of the first image.

18 . The electronic device of claim 15 , wherein the predefined three-dimensional representation comprises a three dimensional model and wherein the second image is not captured by the electronic device.

19 . The electronic device of claim 15 , wherein the instructions further cause the one or more processors to calculate an orientation of the first image based, at least in part, on the matching, wherein the orientation corresponds to a direction from which the first image was captured.

20 . The electronic device of claim 15 , wherein the instructions further cause the one or more processors to add tags based on the one or more features of the identified predefined three-dimensional representation, and wherein the presented composited image includes the added tags.

Assignees

• Apple Inc

Inventors

• Solem Jan Erik
• Holland Jerremy

Classifications

• G06T7/74 Primary

  involving reference images or patches · CPC title

• G06T2207/30244

  Camera pose · CPC title

• G06F16/29

  Geographical information databases · CPC title

• G06K9/68 Primary

  Physics · mapped topic