Image processing apparatus and method

Patent metadata

Field	Value
Publication number	US-10614583-B2
Application number	US-201715494014-A
Country	US
Kind code	B2
Filing date	Apr 21, 2017
Priority date	Oct 31, 2016
Publication date	Apr 7, 2020
Grant date	Apr 7, 2020

Abstract

An image processing apparatus includes a first processor configured to obtain, from a color image, an illumination element image and an albedo element image corresponding to the color image, and a second processor configured to divide the illumination element image into a plurality of subelement images each corresponding to the color image.

First claim

What is claimed is:

1. An image processing apparatus comprising:
a first processor configured to obtain, from a color image, an illumination element image and an albedo element image corresponding to the color image; and
a second processor configured to divide the illumination element image into subelement images each corresponding to the color image, based on a determined normal vector change rate between neighboring pixels in at least one area of the color image.

2. The apparatus of claim 1 , further comprising a memory configured to store instructions;
wherein the first processor is further configured to execute the instructions to obtain, from the color image, the illumination element image and the albedo element image corresponding to the color image; and
wherein the second processor is further configured to execute the instructions to divide the illumination element image into the subelement images each corresponding to the color image.

3. The apparatus of claim 2 , wherein the second processor is further configured to divide the illumination element image into the subelement images using a depth image corresponding to the color image.

4. The apparatus of claim 3 , wherein the second processor is further configured to calculate a normal vector corresponding to each pixel in the at least one area of the color image using the depth image for the dividing.

5. The apparatus of claim 4 , wherein the second processor is further configured to divide the illumination element image into the subelement images based on comparing an illumination element change rate between the neighboring pixels to the normal vector change rate between the neighboring pixels.

6. The apparatus of claim 5 , wherein the second processor is further configured to divide an illumination element change between the neighboring pixels into a shading element and a shadow element based on a degree of correlation between the illumination element change rate between the neighboring pixels and the normal vector change rate between the neighboring pixels.

7. The apparatus of claim 4 , wherein the second processor is further configured to calculate a spherical harmonic function indicating a corresponding relationship between the normal vector and a shading element from an illumination element change between the neighboring pixels, and to divide the illumination element change between the neighboring pixels into the shading element and a shadow element based on a degree of correlation between the illumination element change and the spherical harmonic function.

8. The apparatus of claim 2 , wherein the subelement images include any one or combination of a shading element image, a shadow element image, a highlight element image, and a specular element image.

9. The apparatus of claim 8 , further comprising:
a third processor configured to estimate a direction of light or a position of at least one light source of the color image using the shadow element image.

10. The apparatus of claim 8 , further comprising:
a fourth processor configured to refine a depth image corresponding to the color image using the shading element image.

11. An image processing apparatus comprising:
a first processor configured to obtain, from a color image, an illumination element image and an albedo element image corresponding to the color image; and
a second processor configured to calculate a normal vector corresponding to each pixel of the illumination element image using a depth image corresponding to the color image, cluster the illumination element image to at least one pixel group based on a normal vector change rate between neighboring pixels in the illumination element image, and to divide the illumination element image into subelement images based on comparing an illumination element change rate between neighboring pixels in the at least one pixel group to a normal vector change rate of the pixel group.

12. The apparatus of claim 11 , further comprising a memory configured to store instructions;
wherein the first processor is further configured to execute the instructions to obtain, from the color image, the illumination element image and the albedo element image corresponding to the color image; and
wherein the second processor is further configured to execute the instructions to calculate the normal vector corresponding to each pixel of the illumination element image using the depth image corresponding to the color image, cluster the illumination element image to at least one pixel group based on the normal vector change rate between neighboring pixels in the illumination element image, and to divide the illumination element image into subelement images based on the result of comparing the illumination element change rate between neighboring pixels in the at least one pixel group to the normal vector change rate of the pixel group.

13. The apparatus of claim 12 , wherein the second processor is further configured to divide an illumination element change into a shading element and a shadow element based on a degree of correlation between the illumination element change rate between the neighboring pixels and the normal vector change rate of the pixel group.

14. The apparatus of claim 12 , wherein the second processor is further configured to calculate a spherical harmonic function indicating a corresponding relationship between the normal vector and a shading element from an illumination element change between the neighboring pixels and to divide the illumination element change between the neighboring pixels into the shading element and a shadow element based on a degree of correlation between the illumination element change and the spherical harmonic function.

15. The apparatus of claim 12 , wherein the subelement images include any one or combination of a shading element image, a shadow element image, a highlight element image, and a specular element image.

16. An image processing method comprising:
obtaining, from a color image, an illumination element image and an albedo element image corresponding to the color image; and
dividing the illumination element image into subelement images each corresponding to the color image, based on a determined normal vector change rate between neighboring pixels in at least one area of the color image.

17. The method of claim 16 , wherein the dividing of the illumination element image comprises:
calculating a normal vector corresponding to each pixel in the at least one area of the color image using a depth image corresponding to the color image for determining the normal vector change rate between the neighboring pixels in the at least one area.

18. The method of claim 17 , wherein the dividing of the illumination element image comprises dividing an illumination element change between the neighboring pixels into a shading element and a shadow element based on a degree of correlation between an illumination element change rate between the neighboring pixels and the normal vector change rate between the neighboring pixels.

19. The method of claim 17 , wherein the dividing of the illumination element image comprises:
calculating a spherical harmonic function indicating a corresponding relationship between the normal vector and a shading element from an illumination element change between the neighboring pixels; and
dividing the illumination element change between the neighboring pixels into the shading element and a shadow element based on a degree of correlation between the illumination element change and the spherical harmonic function.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Ha Inwoo
• Lee Hyong Euk
• Sung Young Hun
• Ahn Minsu

Classifications

• G06T2207/20021

  Dividing image into blocks, subimages or windows · CPC title

• G06T2207/10024

  Color image · CPC title

• G06T7/11

  Region-based segmentation · CPC title

• G06T15/50

  Lighting effects · CPC title

• G06T7/507 Primary

  from shading (G06T7/586 takes precedence) · CPC title