Efficient stereo to multiview rendering using interleaved rendering

Patent metadata

Field	Value
Publication number	US-9565414-B2
Application number	US-201314133518-A
Country	US
Kind code	B2
Filing date	Dec 18, 2013
Priority date	May 24, 2013
Publication date	Feb 7, 2017
Grant date	Feb 7, 2017

Abstract

Approaches are described for generating a multiview autostereoscopic image from a stereo three-dimensional input image pair. A stereo to multiview rendering system receives a stereo three-dimensional image including a left image and a right image at a first view position and a second view position, respectively. The system generates a first input warp and a second input warp that maps the left image and the right image to a third and fourth positions, respectively, where the third and fourth positions lie between the first and second view positions. The system generates a plurality of output warps based on the first warp and the second warp. The system resamples each output warp in the plurality of output warps to create a plurality of partial output images. The system interleaves the plurality of partial output images to generate a composite output image.

First claim

What is claimed is:

1. A method for generating a multiview autostereoscopic image, comprising:
receiving a stereo three-dimensional image comprising a left image at a first view position and a right image at a second view position;
generating a first input warp that maps the left image to a third view position that lies between the first view position and the second view position;
generating a second input warp that maps the right image to a fourth view position that lies between the first view position and the second view position;
generating a plurality of output warps based on the first input warp and the second input warp;
resampling each output warp in the plurality of output warps to create a plurality of partial output images; and
interleaving the plurality of partial output images to generate a composite output image.

2. The method of claim 1 , wherein the third view position is equal to the fourth view position.

3. The method of claim 1 , wherein at least one output warp in the plurality of output warps lies between the first view position and the second view position.

4. The method of claim 1 , wherein at least one output warp in the plurality of output warps lies outside of a region between the first view position and the second view position.

5. The method of claim 1 , wherein at least one output warp is generated by linear interpolation of the first input warp and the second input warp.

6. The method of claim 1 , wherein at least one output warp is generated by linear extrapolation of the first input warp and the second input warp.

7. The method of claim 1 , wherein resampling comprises:
transforming pixels of the left image and the right image according to an output warp via a low order interpolation kernel to generate pixels in each corresponding partial output image; and
applying a high quality post-filter to each corresponding partial output image.

8. The method of claim 7 , further comprising performing a locally adaptive super-sampling of each partial output image at a resolution that is higher than a resolution of the respective partial output image.

9. The method of claim 1 , wherein resampling comprises:
applying a high quality pre-filter to the left image to generate a corresponding oversampled left image;
applying a high quality pre-filter to the right image to generate a corresponding oversampled right image; and
transforming pixels of the oversampled left image and the oversampled right image according to an output warp via a low order interpolation kernel in conjunction with locally adaptive supersampling in order to generate pixels in each corresponding partial output image in the plurality of output images.

10. The method of claim 1 , wherein resampling comprises:
applying a high quality pre-filter to the left image to generate a corresponding oversampled left image;
applying a high quality pre-filter to the right image to generate a corresponding oversampled right image; and
transforming pixels of the oversampled left image and the oversampled right image according to an output warp via locally adaptive splatting in order to generate pixels in each corresponding partial output image in the plurality of output images.

11. The method of claim 1 , further comprising storing an animation file on a computer-readable medium, wherein the animation file comprises the output of the receiving, generating, resampling and interleaving steps.

12. A non-transitory computer-readable medium including instructions that, when executed by a processor, cause the processor to generate a multiview autostereoscopic image, by performing an operation comprising:
receiving a stereo three-dimensional image comprising a left image at a first view position and a right image at a second view position;
generating a first input warp that maps the left image to a third view position that lies between the first view position and the second view position;
generating a second input warp that maps the right image to a fourth view position that lies between the first view position and the second view position;
generating a plurality of output warps based on the first input warp and the second input warp;
resampling each output warp in the plurality of output warps to create a plurality of partial output images; and
interleaving the plurality of partial output images to generate a composite output image.

13. The non-transitory computer-readable medium of claim 12 , wherein the third view position is equal to the fourth view position.

14. The non-transitory computer-readable medium of claim 12 , wherein at least one output warp is generated by linear interpolation of the first input warp and the second input warp.

15. The non-transitory computer-readable medium of claim 12 , wherein at least one output warp is generated by linear extrapolation of the first input warp and the second input warp.

16. The non-transitory computer-readable medium of claim 12 , wherein resampling comprises:
transforming pixels of the left image and the right image according to an output warp via a low order interpolation kernel to generate pixels in each corresponding partial output image; and
applying a high quality post-filter to each corresponding partial output image.

17. The non-transitory computer-readable medium of claim 16 , the operation further comprising performing a locally adaptive super-sampling of each partial output image at a resolution that is higher than a resolution of the respective partial output image.

18. The non-transitory computer-readable medium of claim 12 , wherein resampling comprises:
applying a high quality pre-filter to the left image to generate a corresponding oversampled left image;
applying a high quality pre-filter to the right image to generate a corresponding oversampled right image; and
transforming pixels of the oversampled left image and the oversampled right image according to an output warp via a low order interpolation kernel in conjunction with locally adaptive supersampling in order to generate pixels in each corresponding partial output image in the plurality of output images.

19. The non-transitory computer-readable medium of claim 12 , wherein resampling comprises:
applying a high quality pre-filter to the left image to generate a corresponding oversampled left image;
applying a high quality pre-filter to the right image to generate a corresponding oversampled right image; and
transforming pixels of the oversampled left image and the oversampled right image according to an output warp via locally adaptive splatting in order to generate pixels in each corresponding partial output image in the plurality of output images.

20. A computing system, comprising:
a memory that is configured to store instructions for a program; and
a processor that is configured to generate a multiview autostereoscopic image, by performing an operation comprising:
receiving a stereo three-dimensional image comprising a left image at a first view position and a right image at a second view position;
generating a first input warp that maps the left image to a third view position that lies between the first view position and the second view position;
generating a second input warp that maps the right image to a fourth view position that lies between the first view position and the second view position;
generating a plurality of output warps based on the first input warp and the second input warp;
resampling each output warp in the plurality of output warps to create a plurality of pa

Assignees

• Disney Entpr Inc
• Eth Zurich (Eidgenoessische Technische Hochschule Zurich)

Inventors

• Smolic Aljosa Aleksej Andrej
• Greisen Pierre
• Heinzle Simon
• Schaffner Michael
• Gürkaynak Frank Ka{Hacek Over (G)}An

Classifications

• H04N13/111 Primary

  Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation · CPC title

• H04N13/139

  Format conversion, e.g. of frame-rate or size · CPC title

• H04N13/0011 Primary

  Electricity · mapped topic

• H04N13/0029

  Electricity · mapped topic