Image merging method using viewpoint transformation and system therefor

Patent metadata

Field	Value
Publication number	US-2019166350-A1
Application number	US-201816201906-A
Country	US
Kind code	A1
Filing date	Nov 27, 2018
Priority date	Nov 29, 2017
Publication date	May 30, 2019
Grant date	—

Abstract

An image merging method using viewpoint transformation and a system therefor are provided. The method includes obtaining images captured by a plurality of cameras included in the camera system, performing viewpoint transformation for each of the images using a depth map for the images, and merging the images, the viewpoint transformation of which is performed.

First claim

What is claimed is:

1 . An image merging method using viewpoint transformation executed by a computer included in a camera system, the method comprising:
obtaining images captured by a plurality of cameras included in the camera system; performing viewpoint transformation for each of the images using a depth map for the images; and merging the images, the viewpoint transformation of which is performed.

2 . The method of claim 1 , wherein the performing of the viewpoint transformation for each of the images comprises:
setting a viewpoint transformation formula based on a focal length of each of the plurality of cameras, a length of a baseline of the camera system, and parallax between the plurality of cameras, based on the depth map; determining a movement parameter in the viewpoint transformation formula, the movement parameter being a value associated with a distance to move when a location where each of the images is captured moves to a virtual specific location; and transforming viewpoints of the images using the viewpoint transformation formula to be matched.

3 . The method of claim 2 , wherein the determining of the movement parameter in the viewpoint transformation formula comprises:
determining the movement parameter based on a distance to move when an optical center of each of the plurality of cameras moves to a virtual reference location of the camera system.

4 . The method of claim 3 , wherein the determining of the movement parameter in the viewpoint transformation formula comprises:
determining the movement parameter based on the length of the baseline of the camera system.

5 . The method of claim 2 , wherein the transforming of the viewpoints of the images to be matched comprises:
transforming the viewpoint of each of the images into a viewpoint of an image captured at a reference location of the camera system.

6 . The method of claim 1 , wherein the performing of the viewpoint transformation for each of the images further comprises:
filling a hole of the depth map before performing the viewpoint transformation for each of the images.

7 . The method of claim 1 , wherein the performing of the viewpoint transformation for each of the images further comprises:
filling a hole of each of the images, the viewpoint transformation of which is performed.

8 . The method of claim 1 , wherein the merging of the images, the viewpoint transformation of which is performed, further comprises:
transforming a viewpoint of the merged image to fill a hole of the merged image.

9 . The method of claim 1 , wherein the obtaining of the images captured by the plurality of cameras comprises:
obtaining the images with different exposure values from the plurality of cameras, and wherein the merging of the images, the viewpoint transformation of which is performed, comprises: merging the images, with the different exposure values, the viewpoint transformation of which is performed, to generate a hyper dynamic range (HDR) image.

10 . The method of claim 1 , wherein the obtaining of the images captured by the plurality of cameras comprises:
obtaining the images with different focus positions from the plurality of cameras, and wherein the merging of the images, the viewpoint transformation of which is performed, comprises: merging the images, with the different focus positions, the viewpoint transformation of which is performed, to generate an all-in-focus image.

11 . An image merging system using viewpoint transformation, implemented with a computer included in a camera system, the system comprising:
at least one processor configured to execute computer-readable instructions, wherein the at least one processor comprises: an image obtaining unit configured to obtain images captured by a plurality of cameras included in the camera system; a viewpoint transformation performing unit configured to perform viewpoint transformation for each of the images using a depth map for the images; and a merging unit configured to merge the images, the viewpoint transformation of which is performed.

12 . An image registration method executed by a computer included in a camera system, the method comprising:
obtaining images captured by a plurality of cameras included in the camera system; setting a viewpoint transformation formula based on a focal length of each of the plurality of cameras, a length of a baseline of the camera system, and parallax between the plurality of cameras, based on a depth map for the images; determining a movement parameter in the viewpoint transformation formula, the movement parameter being a value associated with a distance to move when a location where each of the images is captured moves to a virtual specific location; and registering images by transforming viewpoints of the images using the viewpoint transformation formula to be matched.

13 . The method of claim 12 , wherein the determining of the movement parameter in the viewpoint transformation formula comprises:
determining the movement parameter based on a distance to move when an optical center of each of the plurality of cameras moves to a virtual reference location of the camera system.

14 . An image registration system implemented with a computer included in a camera system, the system comprising:
at least one processor configured to execute computer-readable instructions, wherein the at least one processor comprises: an image obtaining unit configured to obtain images captured by a plurality of cameras included in the camera system; a formula setting unit configured to set a viewpoint transformation formula based on a focal length of each of the plurality of cameras, a length of a baseline of the camera system, and parallax between the plurality of cameras, based on a depth map for the images; a movement parameter determining unit configured to determine a movement parameter in the viewpoint transformation formula, the movement parameter being a value associated with a distance to move when a location where each of the images is captured moves to a virtual specific location; and an image registration unit configured to register the images by transforming viewpoints of the images using the viewpoint transformation formula to be matched.

Assignees

• Ct Integrated Smart Sensors Found

Inventors

• Park Ki Yeong

Classifications

• H04N23/741

  by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors · CPC title

• H04N5/2355

  Electricity · mapped topic

• H04N13/156

  Mixing image signals · CPC title

• H04N13/111 Primary

  Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation · CPC title

• H04N13/239

  using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence) · CPC title