Digital refocusing method

Patent metadata

Field	Value
Publication number	US-9497437-B2
Application number	US-201514684456-A
Country	US
Kind code	B2
Filing date	Apr 13, 2015
Priority date	Dec 3, 2014
Publication date	Nov 15, 2016
Grant date	Nov 15, 2016

Abstract

A digital refocusing method includes: a plurality of images corresponding to multiple views in a scene are obtained, the images include a central view image and at least one non-central view image; a pixel shift or a pixel index shift is performed to the non-central view image; a line scan along a pre-determined linear path is performed to the central view image and the non-central view images to obtain corresponding pixels of the central view image and corresponding pixels of the non-central view images; view interpolation based on the disparities defined in a disparity map is performed, target pixels corresponded to a novel view image are obtained from the corresponding pixels of the central view image and the corresponding pixels of the non-central view according to a target disparity; and a refocused novel view image is obtained by averaging and compositing the target pixels of novel views.

First claim

What is claimed is:

1. A digital refocusing method, comprising:
obtaining a plurality of images corresponding to multiple views in a scene, wherein the images comprise a central view image and at least one non-central view image;
obtaining a disparity map corresponding to the central view image;
performing a pixel shift or a pixel index shift to the non-central view image;
performing a line scan along a pre-determined linear path to the central view image and the non-central view image to obtain corresponding pixels of the central view image and corresponding pixels of the non-central view image;
performing view interpolation based on disparities defined in the disparity map, obtaining target pixels from the corresponding pixels of the central view image and the corresponding pixels of the non-central view image according to a target disparity, wherein the target pixels are corresponded to a novel view; and
obtaining a refocused novel view image by averaging and compositing the target pixels from a plurality of novel views;
wherein the view interpolation comprises:
obtaining a temporary pixel by mapping the corresponding pixels of the central view to the novel view according to the target disparity;
if there is only one target pixel of the novel view between one of the two neighboring temporary pixels, the target pixel is derived by performing linear interpolation on the two temporary pixels;
if some pixel of the central view is occluded in the novel view, the target pixels are derived by directly mapping the corresponding pixels of the non-central view to the novel view using an estimated disparity which is the closest available one on the disparity map.

2. The digital refocusing method of claim 1 , wherein the images of the multiple views in the scene are obtained through a light-field image capturing device.

3. The digital refocusing method of claim 1 , wherein the pre-determined linear path in the line-scan is a horizontal linear path or a vertical linear path.

4. The digital refocusing method of claim 1 , wherein the pre-determined linear path in the line-scan is an oblique linear path.

5. The digital refocusing method of claim 4 , wherein an angle between the oblique linear path and a horizontal virtual line is 45 degrees.

6. A digital refocusing method, comprising:
performing line scans along a first linear path to a central view image and non-central view images in a scene, obtaining first novel view pixels corresponding to first novel views through performing first view interpolation to the corresponding pixels of the central view image and the corresponding pixels of the non-central view images, and then constructing first novel view images by compositing the first novel view pixels;
performing line scans along a second linear path to the central view image and non-central view images, obtaining second novel view pixels corresponding to second novel views through performing second view interpolation to the corresponding pixels of the central view image and the corresponding pixels of the non-central view images, and then constructing second novel view images by compositing the second novel view pixels; and
performing line scans along a third linear path to the first novel view images and the second novel view images, then obtaining third novel view pixels corresponding to third novel views through performing third view interpolation to the corresponding pixels of the first novel view images and the corresponding pixels of the second novel view images, and then constructing third novel view images by compositing the third novel view pixels,
wherein a triangle view distribution is constructed by the first novel view images, the second novel view images and the third novel view images, and the triangle view distribution is expanded to form two-dimensional novel view images covering a one-dimensional view organization.

7. The digital refocusing method of claim 6 , wherein the first linear path is a horizontal linear path, the second linear path is an oblique linear path, and an angle is formed between the first linear path and the second linear path.

8. The digital refocusing method of claim 7 , wherein the angle is 45 degrees.

9. The digital refocusing method of claim 6 , wherein the third linear path is perpendicular to the first linear path.

Assignees

• Nat Univ Tsing Hua

Inventors

• Huang Chao-Tsung
• Chin Jui

Classifications

• G06T3/4007

  based on interpolation, e.g. bilinear interpolation (image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403) · CPC title

• H04N2013/0081

  Depth or disparity estimation from stereoscopic image signals · CPC title

• H04N13/0011

  Electricity · mapped topic

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title

• H04N13/0022 Primary

  Electricity · mapped topic