Apparatus and method for displaying panoramic images

Patent metadata

Field	Value
Publication number	US-9251561-B2
Application number	US-201013392002-A
Country	US
Kind code	B2
Filing date	May 19, 2010
Priority date	Sep 29, 2009
Publication date	Feb 2, 2016
Grant date	Feb 2, 2016

Abstract

When there is a shooting location of another panoramic image in a shooting direction of a panoramic image to be displayed, a marker setting unit associates a marker, indicating the presence of the another panoramic image in the shooting direction, with the panoramic image to be displayed. A mapping processing unit maps the panoramic image to be displayed, which has been associated with markers for the other panoramic images, into a three-dimensional (3D) panoramic space as textures. A 3D image generator generates a 3D panoramic image when the 3D panoramic space is viewed in a specified line of sight, with the shooting location of the panoramic image to be displayed being set as a viewpoint position. An interface receives an instruction from a user concerning the displayed 3D panoramic image.

First claim

The invention claimed is:

1. A panoramic image display apparatus, comprising:
a storage configured to store panoramic images where information on shooting locations and shooting orientations are associated with each other;
an acquisition unit configured to acquire a first panoramic image to be displayed, from the storage;
a marker setting unit configured to associate a marker, indicating the presence and x, y, and z axis coordinates of a second panoramic image in a shooting direction, with the first panoramic image to be displayed, when the second panoramic image has a shooting location that is in the shooting direction of the first panoramic image to be displayed,
wherein the appearance of the marker is based on the proximity of the second panoramic image relative to the first panoramic image, such that the user can identify the distance between the first and second panoramic images based on the appearance of the marker, and a plurality of markers vary in appearance when the proximities associated with said markers are different;
a mapping processing unit configured to map the first panoramic image to be displayed, which is associated with the marker of the second panoramic image, into a three-dimensional (3D) panoramic space as a texture;
a three-dimensional (3D) image generator configured to generate a three-dimensional (3D) panoramic image, when the three-dimensional panoramic image mapped by the mapping processing unit is viewed in a specified line of sight in such a manner as to regard the shooting location of the first panoramic image to be displayed as a viewpoint position and such that the marker is displayed on said first panoramic image to be displayed at a point of said panoramic image corresponding to said shooting location of said second panoramic image;
a display control unit configured to display the three-dimensional panoramic image on a screen; and
an interface configured to receive an instruction from a user regarding the displayed three-dimensional panoramic image.

2. A panoramic image display apparatus according to claim 1 , further comprising a map image generator configured to generate a map image in such a manner that a symbol indicating the presence of a panoramic image is set at a position of the Earth's surface corresponding to the shooting location,
wherein the display control unit displays the map image on the screen, and
wherein, when the interface receives an instruction to select the symbol of the panoramic image of the map image displayed, the interface sets a panoramic image corresponding to the selected symbol, to the first panoramic image to be displayed.

3. A panoramic image display apparatus according to claim 1 , wherein the marker setting unit sets a different display mode for the marker of the second panoramic image associated with the first panoramic image to be displayed, according to the distance between the shooting location of the first panoramic image to be displayed and the shooting location of the second panoramic image.

4. A panoramic image display apparatus according to claim 1 , wherein
the panoramic images stored in the storage are associated with their respective shooting altitudes,
the storage stores panoramic images shot at different altitudes at the same shooting location, and
when the interface receives an instruction to change the altitude, the interface sets a panoramic image, which has the same shooting location but corresponds to a specified location, to a new panoramic image to be displayed.

5. A panoramic image display apparatus according to claim 1 , wherein
the panoramic images stored in the storage are associated with their respective shooting dates and times,
the storage stores panoramic images shot at different shooting dates and times at the same shooting location, and
when the interface receives an instruction to change the date and time, the interface sets a panoramic image, which has the same shooting location but corresponds to a specified date and time, to a new panoramic image to be displayed.

6. A panoramic image display apparatus according to claim 1 , wherein, when the interface receives an instruction to select a marker in the 3D panoramic image, the interface sets another panoramic image corresponding to the selected marker to be a new panoramic image to be displayed in the place of the panoramic image having been viewed.

7. A panoramic image display apparatus according to claim 6 , wherein,
when the another panoramic image corresponding to the selected marker is set as the new panoramic image to be displayed, the interface sets a shooting location of the new panoramic image to be displayed to a new viewpoint position,
the interface sets, to a new line of sight, a direction in which the shooting location of the new panoramic image to be displayed is viewed from a previous panoramic image, and
the 3D image generator generates a 3D panoramic image in a new line of sight, set by the interface, in which the 3D panoramic space is viewed from the new viewpoint position set by the interface.

8. A panoramic image display apparatus according to claim 6 , wherein,
when the another panoramic image corresponding to the selected marker is set as the new panoramic image to be displayed, the interface sets a shooting location of the new panoramic image to be displayed to a new viewpoint position,
the interface sets, to a new line of sight, a direction in which a previous panoramic view is looked back at from the new panoramic image to be displayed, and
the 3D image generator generates a 3D panoramic image in the new line of sight, set by the interface, in which the 3D panoramic space is viewed from the new viewpoint position set by the interface.

9. A panoramic image display apparatus according to claim 1 , wherein: the marker is displayed on said first panoramic image to be displayed at a point of said panoramic image corresponding to said shooting location of said second panoramic image.

10. A panoramic image display apparatus according to claim 9 , wherein, when a user selects a marker in the first panoramic image to be displayed, the display control unit displays on the screen another panoramic image corresponding to the selected marker in the place of the first panoramic image to be displayed.

11. A method for displaying a panoramic image, comprising:
reading, by a processor, a panoramic image at a shooting place as a first panoramic image to be displayed on a screen, from a storage device that stores a plurality of panoramic images where information on shooting locations and shooting orientations are associated with each other;
associating, by the processor, a marker, that indicates the presence and x, y, and z axis coordinates of a second panoramic image in a shooting direction, with the first panoramic image to be displayed, when the second panoramic image has a shooting location that is in the shooting direction of the first panoramic image to be displayed, and displaying thereby, on the screen, the first panoramic to be displayed,
wherein the appearance of the marker is based on the proximity of the second panoramic image relative to the first panoramic image, such that the user can identify the distance between the first and second panoramic images based on the appearance of the marker, and a plurality of markers vary in appearance when the proximities associated with said markers are different
mapping the first panoramic image to be displayed, which is associated with the marker of the second panoramic image, into a three-dimensional (3D) panoramic space as a texture;
generating a three-dimensional (3D) panoramic image, when the three-dimensional panoramic image is viewed in a specified line of

Assignees

• Furumura Kyoko
• Yamagishi Takeshi
• Hirata Shinichi
• Sony Corp
• Sony Computer Entertainment Inc

Inventors

• Furumura Kyoko
• Yamagishi Takeshi
• Hirata Shinichi

Classifications

• G06T3/4038 Primary

  Image mosaicing, e.g. composing plane images from plane sub-images · CPC title

• H04N23/635

  Region indicators; Field of view indicators · CPC title

• H04N23/698

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• H04N23/80

  Camera processing pipelines; Components thereof · CPC title

• G06T3/0062 Primary

  Physics · mapped topic