Image processing apparatus and method for panoramic image using a single camera

Patent metadata

Field	Value
Publication number	US-9378544-B2
Application number	US-201313761344-A
Country	US
Kind code	B2
Filing date	Feb 7, 2013
Priority date	Mar 15, 2012
Publication date	Jun 28, 2016
Grant date	Jun 28, 2016

Abstract

Provided is an apparatus and method for generating a three-dimensional (3D) panoramic image using a single camera. The image processing apparatus may generate a two-dimensional (2D) reference panoramic image using a central area of each of a plurality of input image frames, and may extract a left-eye strip and a right-eye strip of when a similarity calculated in the plurality of input image frames based on the 2D reference panoramic image has a maximum value.

First claim

What is claimed is:

1. An image processing apparatus, comprising:
a processor to control one or more processor-executable units;
a reference generator to generate a two-dimensional (2D) reference panoramic image using a central area of each of a plurality of input image frames; and
a strip extractor to extract, from an input image frame of the plurality of input image frames, a left-eye strip and a right-eye strip when a similarity of the input image frame compared to the 2D reference panoramic image has a maximum value,
wherein the strip extractor extracts the left-eye strip and the right eye strip from the input image frame when an amount of vertical translation in each strip is most similar to a vertical translation of the 2D reference panoramic image, from among the plurality of input image frames.

2. The image processing apparatus of claim 1 , wherein the reference generator generates the 2D reference panoramic image by blending a predetermined central area in each of the plurality of input image frames.

3. The image processing apparatus of claim 1 , wherein the strip extractor comprises:
a left-eye strip extractor to extract the left-eye strip from the input image frame when a similarity calculated from a predetermined left-eye strip extraction position to a vertical direction based on the 2D reference panoramic image has a maximum value; and
a right-eye strip extractor to extract the right-eye strip from the input image frames when a similarity calculated from a predetermined right-eye strip extraction position to a vertical direction based on the 2D reference panoramic image has a maximum value.

4. The image processing apparatus of claim 1 , wherein the strip extractor comprises:
a left-eye strip extractor to extract the left-eye strip having a maximum correlation value through template matching of a predetermined left-eye strip extraction position in an input image frame corresponding to the 2D reference panoramic image, and
a right-eye strip extractor to extract the right-eye strip having a maximum correlation value through template matching of a predetermined right-eye strip extraction position in the input image frame corresponding to the 2D reference panoramic image.

5. The image processing apparatus of claim 1 , wherein the strip extractor extracts the left-eye strip and the right-eye strip based on a correlation between a color value of a predetermined left-eye strip extraction position and a color value of a predetermined right-eye strip extraction position in an input image frame corresponding to the 2D reference panoramic image.

6. The image processing apparatus of claim 1 , wherein the strip extractor extracts the left-eye strip and the right-eye strip having a same width in an input image frame corresponding to the 2D reference panoramic image.

7. The image processing apparatus of claim 1 , further comprising:
a panoramic image generator to generate a left-eye panoramic image and a right-eye panoramic image based on the left-eye strip and the right-eye strip, respectively.

8. The image processing apparatus of claim 7 , further comprising:
a three-dimensional (3D) panoramic image generator to generate a 3D panoramic image based on a disparity between the left-eye panoramic image and the right-eye panoramic image.

9. The image processing apparatus of claim 1 , further comprising:
a processing unit to perform spherical transformation of an image that is input from a photographing device, and to perform image matching by matching a feature point between the spherically transformed images;
a calculating unit to calculate a translation amount between the spherically transformed images;
a first determining unit to determine whether the translation amount satisfies a predetermined target; and
a second determining unit to store the spherically transformed image when the translation amount satisfies the predetermined target, and to determine whether the number of stored spherically transformed images satisfies a value that is used to generate a 3D panoramic image.

10. An image processing apparatus, comprising:
a processor to control one or more processor-executable units;
a global alignment unit to perform image matching of a plurality of input image frames by matching a feature point between the plurality of input image frames; and
a local alignment unit to locally align a right-eye strip and a left-eye strip in a vertical direction in each of the plurality of input image frames using a two-dimensional (2D) reference panoramic image,
wherein the left-eye strip and the right eye strip are extracted from the input image frame when an amount of vertical translation in each strip is most similar to a vertical translation of the 2D reference panoramic image, from among the plurality of input image frames.

11. The image processing apparatus of claim 10 , wherein the 2D reference panoramic image is generated based on a central area of each of the plurality of input image frames.

12. The image processing apparatus of claim 10 , wherein the local alignment unit extracts the left-eye strip and the right-eye strip when a similarity calculated in the plurality of input image frames based on the 2D reference panoramic image has a maximum value.

13. The image processing apparatus of claim 12 , wherein the local alignment unit generates a left-eye panoramic image and a right-eye panoramic image based on the left-eye strip and the right-eye strip.

14. An image processing method, comprising:
generating, by way of a processor, a two-dimensional (2D) reference panoramic image using a central area of each of a plurality of input image frames; and
extracting, from an input image frame of the plurality of input image frames, a left-eye strip and a right-eye strip when a similarity calculated in the input image frame compared to the 2D reference panoramic image has a maximum value,
wherein the extracting comprises extracting the left-eye strip and the right eye strip from the input image frame when an amount of vertical translation in each strip is most similar to a vertical translation of the 2D reference panoramic image, from among the plurality of input image frames.

15. The method of claim 14 , wherein the generating comprises generating the 2D reference panoramic image by blending a predetermined central area in each of the plurality of input image frames.

16. The method of claim 14 , wherein the extracting comprises:
extracting the left-eye strip from the input image frame when a similarity calculated from a predetermined left-eye strip extraction position to a vertical direction based on the 2D reference panoramic image has a maximum value; and
extracting the right-eye strip from the input image frame when a similarity calculated from a predetermined right-eye strip extraction position to a vertical direction based on the 2D reference panoramic image has a maximum value.

17. The method of claim 14 , wherein the extracting comprises:
extracting a left-eye strip having a maximum correlation value through template matching of a predetermined left-eye strip extraction position in an input image frame corresponding to the 2D reference panoramic image; and
extracting a right-eye strip having a maximum correlation value through template matching of a predetermined right-eye strip extraction position in the input image frame corresponding to the 2D reference panoramic image.

18. The method of claim 14 , wherein the extracting comprises extracting the left-eye strip and the right-eye strip based on a correlation between a color value of a pr

Assignees

• Samsung Electronics Co Ltd

Inventors

• Cha Joon Hyuk
• Moon Young Su
• Jeon Young Sun
• Lee Shi Hwa

Classifications

• H04N23/698

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• G06T3/4038 Primary

  Image mosaicing, e.g. composing plane images from plane sub-images · CPC title

• G06T5/50 Primary

  using two or more images, e.g. averaging or subtraction · CPC title

• H04N13/261

  with monoscopic-to-stereoscopic image conversion · CPC title

• H04N13/156 Primary

  Mixing image signals · CPC title