Alternate viewpoint image enhancement

Patent metadata

Field	Value
Publication number	US-10030990-B2
Application number	US-201614997819-A
Country	US
Kind code	B2
Filing date	Jan 18, 2016
Priority date	Jun 28, 2012
Publication date	Jul 24, 2018
Grant date	Jul 24, 2018

Abstract

In one embodiment, panoramic images, images bubbles, or any two-dimensional views of three-dimensional subject matter are enhanced with one or more alternate viewpoints. A controller receives data indicative of a point on the two-dimensional perspective and accesses a three-dimensional location based on the point. The controller selects an image bubble based on the three-dimensional location. The three-dimensional location may be determined according to a depth map corresponding to the point. A portion of the image bubble is extracted and incorporated into the two-dimensional perspective. The resulting image may be a seamless enhanced resolution image or include a picture-in-picture enhanced resolution window including subject matter surrounding the selected point.

First claim

I claim:

1. A method comprising:
receiving first image data for a first image bubble captured from a first perspective;
receiving second image data for a second image bubble captured from a second perspective; and
combining, using a processor, the first image data and the second image data into a composite image, wherein the composite image includes a spatial resolution greater than a resolution of the first image bubble and a resolution of the second image bubble,
wherein one or more of the first image data and second image data comprise image data for multiple viewpoints within the first image bubble or second image bubble, respectively,
wherein the composite image is from a third perspective, and
wherein the composite image comprises an inset window, the inset window including viewpoints otherwise unavailable from the third perspective.

2. The method of claim 1 , further comprising:
receiving a first depth map associated with the first image bubble; and
receiving a second depth map associated with the second image bubble,
wherein the composite image is generated based on the first depth map and the second depth map.

3. The method of claim 1 , further comprising:
identifying a third image bubble, wherein the first image bubble and the second image bubble are combined with the third image bubble to create the composite image.

4. The method of claim 3 , further comprising:
identifying an object of interest in the first image bubble; and
identifying the object of interest in the third image bubble, wherein the object of interest in the first image bubble has a higher resolution than the object of interest in the third image bubble.

5. The method of claim 4 , wherein the object of interest is identified from an image processing algorithm.

6. The method of claim 5 , wherein the image processing algorithm includes edge detection, object recognition, optical character recognition, or feature extraction.

7. The method of claim 1 , further comprising:
receiving a user selection of a point of the second image data; and
transitioning from the first perspective to the second perspective in response to the user selection.

8. The method of claim 1 , wherein at least a portion of the first image bubble includes scenery obstructed from the third perspective.

9. The method of claim 1 , further comprising:
receiving a command to manipulate the composite image by view angle or zoom.

10. The method of claim 1 , wherein the composite image includes image data for multiple viewpoints within the composite image.

11. The method of claim 1 , wherein the third perspective is the same as the first perspective.

12. An apparatus comprising:
a memory configured to store first image data for a first image bubble captured from a first perspective and second image data for a second image bubble captured from a second perspective; and
a processor configured to combine the first image data and the second image data into a composite image, wherein the composite image includes a spatial resolution greater than a resolution of the first image bubble and a resolution of the second image bubble,
wherein the composite image is from a third perspective, and
wherein the composite image comprises an inset window, the inset window including views not otherwise available from the third perspective.

13. The apparatus of claim 12 , wherein the processor is configured to access a first depth map associated with the first image bubble and a second depth map associated with the second image bubble,
wherein the composite image is generated based on the first depth map and the second depth map.

14. The apparatus of claim 13 , wherein the first depth map and the second depth map includes depth values for three-dimensional locations in the first and second image bubbles.

15. The apparatus of claim 12 , wherein the processor is configured to identify a first view of an object of interest in the first image bubble and a second view of the object of interest in the third image bubble, wherein the object of interest in the first image bubble has a higher resolution than the object of interest in the third image bubble.

16. The apparatus of claim 12 , wherein the processor is configured to receive a user selection of a point of the second image data and transition from the first perspective to the second perspective in response to the user selection.

17. The apparatus of claim 12 , wherein the composite image includes image data for multiple viewpoints within the composite image.

18. A method comprising:
receiving first image data for a first image bubble captured from a first perspective, the first perspective comprising a field of view in a first direction;
receiving second image data for a second image bubble captured from a second perspective, the second perspective comprising a field of view in a second direction, the second direction different than the first direction;
combining the first image data and the second image data into a composite image, wherein the composite image comprises an inset window containing the second image data from the second perspective;
receiving a user selection of a point of the second image data; and
transitioning, using a processor, from the first perspective to the second perspective in response to the user selection.

19. The method of claim 18 , further comprising:
combining the first image data and the second image data into a composite image, wherein the composite image includes a spatial resolution greater than a resolution of the first image bubble and a resolution of the second image bubble.

20. An apparatus comprising:
a memory configured to store first image data for a first image bubble captured from a first perspective and second image data for a second image bubble captured from a second perspective, wherein the first perspective comprises a field of view in a first direction and the second perspective comprises a field of view in a second direction, the second direction different than the first direction;
an interface configured to display the second image data from the second perspective in an inset window within a display of the first image data from the first perspective and to receive a user selection of a point of the second image data; and
a processor configured to calculate a transition from the first perspective to the second perspective in response to the user selection.

Assignees

• Here Global Bv

Inventors

• Lynch James D

Classifications

• G06T3/4038

  Image mosaicing, e.g. composing plane images from plane sub-images · CPC title

• G06T11/00

  Two-dimensional [2D] image generation · CPC title

• G06T2200/32

  involving image mosaicing · CPC title

• G06T2210/61

  Scene description · CPC title

• G09B29/007

  using computer methods · CPC title