Fast intrinsic images

Patent metadata

Field	Value
Publication number	US-10297045-B2
Application number	US-201414546934-A
Country	US
Kind code	B2
Filing date	Nov 18, 2014
Priority date	Nov 18, 2014
Publication date	May 21, 2019
Grant date	May 21, 2019

Abstract

Fast intrinsic images techniques are described. In one or more implementations, a combination of local constraints on shading and reflectance and non-local constraints on reflectance are applied to an image to generate a linear system of equations. The linear system of equations can be solved to generate a reflectance intrinsic image and a shading intrinsic image for the image. In one or more implementations, a multi-scale parallelized iterative solver is used to solve the linear system of equations to generate the reflectance intrinsic image and the shading intrinsic image.

First claim

What is claimed is:

1. A method comprising:
receiving an image;
determining, for each pixel in the image, local constraints for both reflectance variations and shading variations between the pixel and neighboring pixels adjacent to the pixel, the local constraints determined based on a value indicating a difference between a reflectance of the pixel and a reflectance of the neighboring pixels;
determining, for each pixel in the image, non-local constraints for reflectance variations between the pixel and pixels in a set of pixels of the image, the set of pixels of the image being different than the neighboring pixels and the non-local constraints being different than the local constraints; and
generating a reflectance intrinsic image for the image and a shading intrinsic image for the image by applying the local constraints for reflectance variations and shading variations and the non-local constraints for reflectance variations to the image.

2. The method of claim 1 , further comprising determining a proxy image for the image, wherein determining the local constraints for reflectance variations and determining shading variations and the non-local constraints for reflectance variations are based at least in part on the proxy image.

3. The method of claim 2 , wherein determining the proxy image is based on an assumption that shading of the image is grayscale.

4. The method of claim 3 , wherein determining the proxy image further comprises determining a chromacity of the image.

5. The method of claim 2 , further comprising:
detecting that shading of the image is colored;
requesting user input in response to detecting that shading of the image is colored; and
determining the proxy image based at least in part on user input received in response to the request.

6. The method of claim 5 , wherein the proxy image comprises an illumination-invariant RGB image.

7. The method of claim 5 , wherein the requesting user input comprises requesting user input to specify two points on the image with the same reflectance where one of the two points is shadowed and the other one of the two points is illuminated.

8. The method of claim 1 , wherein generating the reflectance intrinsic image and the shading intrinsic image is performed by combining the local constraints and the non-local constraints.

9. The method of claim 1 , wherein generating the reflectance intrinsic image and the shading intrinsic image is performed using a direct linear solver.

10. The method of claim 1 , wherein generating the reflectance intrinsic image and the shading intrinsic image is performed using a multi-scale parallel iterative solver.

11. The method of claim 10 , wherein the multi-scale parallel iterative solver uses Jacobi iterations to generate the reflectance intrinsic image and the shading intrinsic image.

12. A method comprising:
receiving an image;
determining a reflectance intrinsic image for the image and a shading intrinsic image for the image by:
down-sampling the image to a coarsest level;
generating a reflectance intrinsic image and a shading intrinsic image for the down-sampled image based on a combination of:
local constraints for both reflectance variations and shading variations between an area of the down-sampled image and neighboring areas adjacent to the area, the local constraints being based on a value indicating a difference between a reflectance of the area and a reflectance of the neighboring areas; and
non-local constraints for reflectance among areas of the down-sampled image, the local constraints being different than the non-local constraints;

for at least one iteration, up-sampling the down-sampled image, and generating a reflectance intrinsic image and a shading intrinsic image for the up-sampled image; and
responsive to determining that the down-sampled image has been up-sampled to an original resolution of the image, generating the reflectance intrinsic image for the image and the shading intrinsic image for the image based on the original resolution of the image; and

outputting the determined reflectance intrinsic image and the shading intrinsic image.

13. The method of claim 12 , wherein Jacobi iterations are used for the at least one iteration to generate the reflectance intrinsic image and the shading intrinsic image for the up-sampled image.

14. The method of claim 12 , wherein generating the reflectance intrinsic image for the image and the shading intrinsic image for the image includes determining a log shading value for the image.

15. The method of claim 14 , further comprising determining the shading intrinsic image by exponentiating the log shading value.

16. The method of claim 15 , further comprising determining the reflectance intrinsic image by dividing the image by the shading intrinsic image.

17. A system comprising:
a processor; and
at least one computer readable storage media having instructions stored thereon that are executable by the processor to perform operations comprising:
receiving an image;
generating a reflectance intrinsic image for the image and a shading intrinsic image for the image by applying an intrinsic images model to the image, the intrinsic images model comprising local constraints for both reflectance variations and shading variations between a pixel of the image and neighboring pixels adjacent to the pixel, and non-local constraints for reflectance variations among pixels of the image, the local constraints being different from the non-local constraints and based on a value indicating a difference between a reflectance of the pixel and a reflectance of the neighboring pixels; and
outputting the generated reflectance intrinsic image and shading intrinsic image.

18. The system of claim 17 , wherein generating the reflectance intrinsic image for the image and the shading intrinsic image for the image comprises:
down-sampling the image to a coarsest level;
determining a shading value for the down-sampled image; and
refining the shading value by up-sampling the image, for one or more iterations, and generating a reflectance intrinsic image for the up-sampled image and a shading intrinsic image for the up-sampled image using Jacobi iterations.

19. The system of claim 17 , the operations further comprising:
generating an edited reflectance intrinsic image in response to receiving user input indicating a change to the reflectance intrinsic image for the image; and
generating an edited image by combining the edited reflectance intrinsic image with the shading intrinsic image for the image.

20. The system of claim 17 , wherein the value indicating the difference between the reflectance of the pixel and the reflectance of the neighboring pixels is inversely proportional to the difference between the reflectance of the pixel and the reflectance of the neighboring pixels.

Assignees

• Adobe Inc

Inventors

• Sunkavalli Kalyan K

Classifications

• G06T2207/20016

  Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform · CPC title

• G06T2207/10024

  Color image · CPC title

• G06T7/90 Primary

  Determination of colour characteristics · CPC title

• G06T7/11

  Region-based segmentation · CPC title