Apparatus and method for spatially referencing images

Patent metadata

Field	Value
Publication number	US-9740962-B2
Application number	US-201615133836-A
Country	US
Kind code	B2
Filing date	Apr 20, 2016
Priority date	Mar 19, 2012
Publication date	Aug 22, 2017
Grant date	Aug 22, 2017

Abstract

Provided is a method of spatially referencing a plurality of images captured from a plurality of different locations within an indoor space by determining the location from which the plurality of images were captured. The method may include obtaining a plurality of distance-referenced panoramas of an indoor space. The distance-referenced panoramas may each include a plurality of distance-referenced images each captured from one position in the indoor space and at a different azimuth from the other distance-referenced images, a plurality of distance measurements, and orientation indicators each indicative of the azimuth of the corresponding one of the distance-referenced images. The method may further include determining the location of each of the distance-referenced panoramas based on the plurality of distance measurements and the orientation indicators and associating in memory the determined locations with the plurality of distance-referenced images captured from the determined location.

First claim

The invention claimed is:

1. A method of determining the location from which images were captured, the method comprising:
obtaining, by one or more processors, a plurality of distance-referenced images, each distance-referenced image captured from a different position among a plurality of positions;
determining, by the one or more processors, locations of the plurality of positions in a shared coordinate system by calculating a difference between a first distance and a second distance, wherein the first distance is measured from a first position among the plurality of positions in an orientation, and the second distance is measured from a second position among the plurality of positions in a same orientation as the first distance; and
associating, by the one or more processors, the determined locations with corresponding images of the distance-referenced images in spatially-referenced images stored by a spatially-referenced image repository.

2. The method of claim 1 , comprising:
receiving, by the one or more processors, a request for a spatially-referenced image user interface;
retrieving, by the one or more processors, based on the request, from the spatially-referenced image repository a spatially-referenced image responsive to the request;
identifying, by the one or more processors, based on a position of the spatially-referenced image, another spatially-referenced image stored in the spatially-referenced image repository;
transmitting, by the one or more processors, the spatially-referenced image and a selectable icon by which a user requests the other spatially-referenced image.

3. The method of claim 1 , wherein the locations of the plurality of positions in the shared coordinate system are relative to a point of reference, the point of reference being either absolute or relative.

4. The method of claim 1 , wherein obtaining the plurality of distance-referenced images includes placing a distance-referencing camera at each of the plurality of positions and orienting the distance-referencing camera at the same orientation during capture of each distance-referenced image.

5. The method of claim 4 , further comprising taking an orientation measurement of the distance-referencing camera at each of the plurality of positions to confirm the same orientation at each position.

6. The method of claim 1 , wherein the associating the determined locations with corresponding images of the distance-referenced images in the spatially-referenced images includes converting the plurality of distance-referenced images into corresponding spatially-referenced panoramas.

7. The method of claim 6 , further comprising preparing one or more of the spatially-referenced panoramas for presentation to a user.

8. The method of claim 1 , further comprising determining a distance between two perspectives from a pair of the spatially-referenced images.

9. The method of claim 1 , wherein the spatially-referenced images are spatially-referenced panoramas, and the method further comprises constructing a three-dimensional model of an indoor space based on a stereoscopic view of features from two of the spatially-referenced panoramas.

10. The method of claim 9 , further comprising texture mapping the two spatially-referenced panoramas onto the three-dimensional model of the indoor space.

11. A system for acquiring spatially-referenced images of indoor spaces, the camera comprising:
a camera having an optical axis;
a camera mount coupled to the camera and configured to align the optical axis with a plurality of generally radially symmetric directions, each of the radially symmetric directions approximately corresponding to an angular field of view of the camera;
a laser range finders coupled to the camera and oriented approximately parallel to the optical axis of the camera;
an orientation sensor coupled to the camera and configured to indicate which of the radially symmetric directions the camera is facing;
one or more processors communicatively connected to the laser range finders and the camera; and
non-transitory machine-readable memory communicatively connected to the one or more processors, the memory storing instructions that when executed by the one or more processors cause the one or more processors to:
obtain a plurality of distance-referenced images, each distance-referenced image captured from a different position among a plurality of positions;
determine locations of the plurality of positions in a shared coordinate system by calculating a difference between a first distance and a second distance, wherein the first distance is measured from a first position among the plurality of positions in an orientation, and the second distance is measured from a second position among the plurality of positions in a same orientation as the first distance; and
associate the determined locations with corresponding images of the distance-referenced images in spatially-referenced images stored by a spatially-referenced image repository.

12. The system of claim 11 , wherein the system further includes a tour server configured to:
receive a request for a spatially-referenced image user interface;
retrieve based on the request, from the spatially-referenced image repository a spatially-referenced image responsive to the request;
identify, based on a position of the spatially-referenced image, another spatially-referenced image stored in the spatially-referenced image repository; and
transmit the spatially-referenced image and a selectable icon by which a user requests the other spatially-referenced image.

13. The system of claim 11 , wherein the plurality of distance-referenced images are obtained by placing the camera at each of the plurality of positions and orienting the camera at the same orientation during capture of each distance-referenced image.

14. The system of claim 13 , wherein the orientation sensor is configured to take an orientation measurement of the camera at each of the plurality of positions to confirm the same orientation at each position.

15. The system of claim 11 , wherein the system further includes a tour server configured to prepare one or more of the spatially-referenced images for presentation to a user.

16. The system of claim 11 , wherein the system further includes a tour server configured to determine a distance between two perspectives from a pair of the spatially-referenced images.

17. The system of claim 11 , wherein the spatially-referenced images are spatially-referenced panoramas, and the system further includes a tour server configured to construct a three-dimensional model of an indoor space based on a stereoscopic view of features from two of the spatially-referenced panoramas.

18. A non-transitory computer-readable storage medium having instructions stored thereon, the instructions, when executed by one or more processors, cause the one or more processors to perform a method of determining a location from which images were captured, the method comprising:
obtaining a plurality of distance-referenced images, each distance-referenced image captured from a different position among a plurality of positions;
determining locations of the plurality of positions in a shared coordinate system by calculating a difference between a first distance and a second distance, wherein the first distance is measured from a first position among the plurality of positions in an orientation, and the second distance is measured from a second position among the plurality of positions in a same orientation as the first distance; and
associating the determined

Assignees

• Google Inc

Inventors

• Starns Alexander Thomas
• Raman Arjun
• Royz Gadi

Classifications

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• H04N13/20 Primary

  Image signal generators · CPC title

• H04N23/698

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• G06T2207/30244

  Camera pose · CPC title

• G06T2207/10012

  Stereo images · CPC title