Reducing view transitions artifacts in automultiscopic displays

Patent metadata

Field	Value
Publication number	US-9967538-B2
Application number	US-201514613924-A
Country	US
Kind code	B2
Filing date	Feb 4, 2015
Priority date	Nov 4, 2013
Publication date	May 8, 2018
Grant date	May 8, 2018

Abstract

Automultiscopic displays enable glasses-free 3D viewing by providing both binocular and motion parallax. Within the display field of view, different images are observed depending on the viewing direction. When moving outside the field of view, the observed images may repeat. Light fields produced by lenticular and parallax-barrier automultiscopic displays may have repetitive structure with significant discontinuities between the fields of view. This repetitive structure induces visual artifacts in the form of view discontinuities, depth reversals, and extensive disparities. To overcome this problem, a method modifies the presented light field image content and makes it more repetitive. In the method, a light field is refined using global and local shearing and then the repeating fragments are stitched. The method reduces the discontinuities in the displayed light field and leads to visual quality improvements. Benefits of the method are shown using an automultiscopic display with a parallax barrier and lenticular prints.

First claim

What is claimed is:

1. A computer-implemented method comprising:
storing multi-view automultiscopic image content of a multi-view image including an image scene or two-dimensional slice in an electronic memory;
reducing the visibility of one or more artifacts from the multi-view automultiscopic image content by modifying the multi-view automultiscopic image content based upon shearing the multi-view automultiscopic image content globally, followed by shearing the multi-view automultiscopic image content locally, the local shearing further reducing the visibility of the one or more artifacts as compared with the global shearing, and followed by stitching of repeated fragments of the multi-view automultiscopic image content, the stitching including determining an optimal stitch cut in a gradient domain based upon overlapping repeated fragments of the multi-view automultiscopic image content, the stitching performing a further additional reduction of the visibility of the one or more artifacts as compared with the local shearing; and
based upon the modification of the multi-view automultiscopic image content, providing one or more updated multi-view images with improved visibility as compared with the multi-view image, at least with respect to the one or more artifacts.

2. The method of claim 1 , wherein:
the multi-view automultiscopic image content includes the image scene; and
shearing the multi-view automultiscopic image content globally includes repositioning the image scene along a depth plane.

3. The method of claim 2 , wherein repositioning the image scene includes adjusting a slope of one or more lines that pass through an intersection of a line corresponding to a given point in the image scene.

4. The method of claim 1 , wherein shearing the multi-view automultiscopic image content locally includes:
dividing the multi-view automultiscopic image content into a plurality of portions of the multi-view automultiscopic image content, and
repositioning a plurality of views of each of the portions of the multi-view automultiscopic image content.

5. The method of claim 1 , wherein stitching the multi-view automultiscopic image content includes propagating shears of the multi-view automultiscopic image content into the multi-view image using linear interpolation and presenting objects of the multi-view automultiscopic image content at different depths with different viewing locations.

6. The method of claim 1 , wherein the multi-view automultiscopic image content includes multi-view frames across a time domain.

7. The method of claim 6 , further comprising:
periodically selecting a sample of multi-view frames from the time domain;
periodically performing reducing the visibility of the one or more artifacts from the sample of multi-view frames by modifying the multi-view automultiscopic image content; and
periodically performing reducing the visibility of the one or more artifacts from non-selected multi-view frames by interpolating the shearing of the multi-view automultiscopic image content locally and interpolating the shearing of the multi-view automultiscopic image content globally.

8. The method of claim 1 , wherein the global shearing is performed based upon first and last views of the multi-view automultiscopic image content, a total number of pixels of the multi-view image, and a matching error between a neighborhood of a given pixel of the first view and a neighborhood of a corresponding pixel of the last view.

9. A system comprising:
a memory storing multi-view automultiscopic image content of a multi-view image including an image scene or two-dimensional slice;
an artifact removal module configured to reduce the visibility of one or more artifacts from the multi-view automultiscopic image content by modifying the multi-view automultiscopic image content based upon shearing the multi-view automultiscopic image content globally, followed by shearing the multi-view automultiscopic image content locally, the local shearing further reducing the visibility of the one or more artifacts as compared with the global shearing, and followed by stitching of repeated fragments of the multi-view automultiscopic image content, the stitching including determining an optimal stitch cut in a gradient domain based upon overlapping repeated fragments of the multi-view automultiscopic image content, the stitching performing a further additional reduction of the visibility of the one or more artifacts as compared with the local shearing; and
the artifact removal module, based upon the modification of the multi-view automultiscopic image content, being further configured to provide one or more updated multi-view images with improved visibility as compared with the multi-view image, at least with respect to the one or more artifacts.

10. The system of claim 9 , wherein the multi-view automultiscopic image content includes the image scene, and shearing the multi-view automultiscopic image content globally includes repositioning the image scene along a depth plane.

11. The system of claim 10 , wherein repositioning the image scene includes adjusting a slope of one or more lines that pass through an intersection of a line corresponding to a given point in the image scene.

12. The system of claim 9 , wherein shearing the multi-view automultiscopic image content locally includes:
a) dividing the multi-view automultiscopic image content into a plurality of portions of the multi-view automultiscopic image content, and
b) repositioning a plurality of views of each of the portions of the multi-view automultiscopic image content.

13. The system of claim 9 , wherein stitching the multi-view automultiscopic image content includes propagating shears of the multi-view automultiscopic image content into the multi-view image using linear interpolation and presenting objects of the multi-view automultiscopic image content at different depths with different viewing locations.

14. The system of claim 9 , wherein the multi-view automultiscopic image content includes multi-view frames across a time domain.

15. The system of claim 14 , further comprising:
a selection module configured to periodically select a sample of multi-view frames from the time domain;
wherein the artifact removal module is configured to periodically reduce the visibility of the one or more artifacts from the sample of multi-view frames by modifying the multi-view automultiscopic image content and is configured to periodically reduce the visibility of the one or more artifacts from non-selected multi-view frames by interpolating the shearing of the multi-view automultiscopic image content locally and interpolating the shearing of the multi-view automultiscopic image content globally.

16. A non-transitory computer readable medium having stored thereon a sequence of instructions which, when loaded and executed by a processor coupled to an apparatus, causes the apparatus to:
store multi-view automultiscopic image content including an image scene or two-dimensional slice;
reduce the visibility of one or more artifacts from the multi-view automultiscopic image content by modifying the multi-view automultiscopic image content based upon shearing the multi-view automultiscopic image content globally, followed by shearing the multi-view automultiscopic image content locally, the local shearing further reducing the visibility of the one or more artifacts as compared with the global shearing, and followed by stitching of repeated fragments of the multi-view automultiscopic image content, the stitching including determining an optimal stitch cut in a gradient d

Assignees

• Massachusetts Inst Technology
• Massachussetts Institute Of Tech

Inventors

• Didyk Piotr Krzysztof
• Du Song-Pei
• Durand Frederic
• Matusik Wojciech

Classifications

• H04N13/0445

  Electricity · mapped topic

• H04N13/0018 Primary

  Electricity · mapped topic

• H04N13/0402

  Electricity · mapped topic

• H04N13/122 Primary

  Improving the three-dimensional [3D] impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues (H04N13/128 takes precedence) · CPC title

• H04N13/302

  for viewing without the aid of special glasses, i.e. using autostereoscopic displays · CPC title