Methods and apparatus for coherent manipulation and stylization of stereoscopic images

Patent metadata

Field	Value
Publication number	US-9105117-B2
Application number	US-201213675455-A
Country	US
Kind code	B2
Filing date	Nov 13, 2012
Priority date	Nov 30, 2011
Publication date	Aug 11, 2015
Grant date	Aug 11, 2015

Abstract

Methods and apparatus for coherent manipulation and stylization of stereoscopic images. A stereo image manipulation method may use the disparity map for a stereo image pair to divide the left and right images into a set of slices, each of which is the portion of the images that correspond to a certain, small depth range. The method may merge the left and right slices for a depth into a single image. The method may then apply a stylization technique to each slice. The method may then extract the left and right portions of each stylized slice, and stack them together to create a coherent stylized stereo image. As an alternative to first extracting slices from a merged image and then applying a stylization technique to the slices, the method may first apply the stylization technique to the merged image and then extract slices from the stylized merged image.

First claim

What is claimed is:

1. A method, comprising:
performing, by one or more computing devices:
receiving a stereo image pair comprising a left image and a right image;
responsive to receiving the stereo image pair, obtaining a disparity map for the stereo image pair based on differences between the left image and the right image;
for each of a plurality of depth ranges indicated by the disparity map, generating a merged image that includes image data from the left image and image data from the right image at the respective depth range; and
generating a new stereo image pair comprising a new left image and a new right image from one or more of the plurality of merged images.

2. The method as recited in claim 1 , further comprising applying a stylization technique to at least one of the one or more merged images prior to said generating a new stereo image pair.

3. The method as recited in claim 2 , wherein said applying a stylization technique to at least one of the one or more merged images comprises, for each of the at least one of the one or more merged images:
extracting a slice image from the respective merged image at a particular depth as indicated by the disparity map; and
applying the stylization technique to the slice image to generate a stylized slice image.

4. The method as recited in claim 2 , wherein said applying a stylization technique to at least one of the one or more merged images comprises, for each of the at least one of the one or more merged images:
applying the stylization technique to the respective merged image to generate a stylized merged image; and
extracting a stylized slice image from the stylized merged image at a particular depth.

5. The method as recited in claim 2 , further comprising applying a different stylization technique to at least one other of the one or more merged images prior to said generating a new stereo image pair.

6. The method as recited in claim 1 , wherein said generating a new stereo image pair comprising a new left image and a new right image from one or more of the plurality of merged images comprises:
extracting a slice image from each of the one or more merged images at a particular depth as indicated by the disparity map;
extracting a leftmost portion from each of the one or more slice images and stacking the one or more leftmost portions to generate the new left image; and
extracting a rightmost portion from each of the one or more slice images and stacking the one or more rightmost portions to generate the new right image.

7. The method as recited in claim 1 , further comprising:
extracting a slice image from each of the one or more merged images at a particular depth as indicated by the disparity map;
extracting a portion from each of the one or more slice images and stacking the one or more portions to generate an intermediate image between the left image and the right image.

8. The method as recited in claim 1 , wherein the disparity map indicates a disparity for each pixel in the stereo image pair, wherein each disparity indicates an offset from a corresponding pixel in the left image or the right image, and wherein each disparity corresponds to a particular depth in the stereo image pair.

9. A system, comprising:
one or more processors; and
a memory comprising program instructions, wherein the program instructions are executable by the at least one of the one or more processors to:
receive a stereo image pair comprising a left image and a right image;
responsive to receiving the stereo image pair, obtain a disparity map for the stereo image pair based on differences between the left image and the right image;
for each of a plurality of depth ranges indicated by the disparity map, generate a merged image that includes image data from the left image and image data from the right image at the respective depth range; and
generate a new stereo image pair comprising a new left image and a new right image from one or more of the plurality of merged images.

10. The system as recited in claim 9 , wherein the program instructions are further executable by the at least one of the one or more processors to apply a stylization technique to at least one of the one or more merged images prior to said generating a new stereo image pair.

11. The system as recited in claim 10 , wherein, to apply a stylization technique to at least one of the one or more merged images, the program instructions are executable by the at least one of the one or more processors to, for each of the at least one of the one or more merged images:
extract a slice image from the respective merged image at a particular depth as indicated by the disparity map; and
apply the stylization technique to the slice image to generate a stylized slice image.

12. The system as recited in claim 10 , wherein, to apply a stylization technique to at least one of the one or more merged images, the program instructions are executable by the at least one of the one or more processors to, for each of the at least one of the one or more merged images:
apply the stylization technique to the respective merged image to generate a stylized merged image; and
extract a stylized slice image from the stylized merged image at a particular depth.

13. The system as recited in claim 10 , wherein the program instructions are executable by the at least one of the one or more processors to apply a different stylization technique to at least one other of the one or more merged images prior to said generating a new stereo image pair.

14. The system as recited in claim 9 , wherein, to generate a new stereo image pair comprising a new left image and a new right image from one or more of the plurality of merged images, the program instructions are executable by the at least one of the one or more processors to:
extract a slice image from each of the one or more merged images at a particular depth as indicated by the disparity map;
extract a leftmost portion from each of the one or more slice images and stack the one or more leftmost portions to generate the new left image; and
extract a rightmost portion from each of the one or more slice images and stack the one or more rightmost portions to generate the new right image.

15. A non-transitory computer-readable storage medium storing program instructions, wherein the program instructions are computer-executable to implement:
receiving a stereo image pair comprising a left image and a right image;
responsive to receiving the stereo image pair, obtaining a disparity map for the stereo image pair based on differences between the left image and the right image;
for each of a plurality of depth ranges indicated by the disparity map, generating a merged image that includes image data from the left image and image data from the right image at the respective depth range; and
generating a new stereo image pair comprising a new left image and a new right image from one or more of the plurality of merged images.

16. The non-transitory computer-readable storage medium as recited in claim 15 , wherein the program instructions are further computer-executable to implement applying a stylization technique to at least one of the one or more merged images prior to said generating a new stereo image pair.

17. The non-transitory computer-readable storage medium as recited in claim 16 , wherein, in said applying a stylization technique to at least one of the one or more merged images, the program instructions are computer-executable to implement, for each of the at least one of the one or more merged images:
extracting a slice im

Assignees

• Adobe Systems Inc

Inventors

• Asente Paul
• Northam Lesley Ann

Classifications

• G06T11/10 Primary

  Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) · CPC title

• G06T19/20

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G06T2219/2024

  Style variation · CPC title

• G06T15/00 Primary

  Three-dimensional [3D] image rendering · CPC title

• G06T11/001

  Physics · mapped topic