Image processing apparatus, image capture apparatus, and control method for adding an effect of a virtual light source to a subject

Patent metadata

Field	Value
Publication number	US-10694111-B2
Application number	US-201816203914-A
Country	US
Kind code	B2
Filing date	Nov 29, 2018
Priority date	Feb 18, 2016
Publication date	Jun 23, 2020
Grant date	Jun 23, 2020

Abstract

With respect to a subject included in an image, the illuminating condition by an ambient light source in an environment where the image was captured is estimated, and based on the estimation result, the effect of a virtual light source that was non-existent at the time of image capture is computed. More specifically, the effect of the virtual light source is computed using an illumination direction of the virtual light source and the reflective characteristics of the subject illuminated by the virtual light source, which have been determined based on the estimation result, and an image derived from addition of the effect of the virtual light source is output.

First claim

What is claimed is:

1. An image processing apparatus, comprising:
a processor; and
a memory including instructions that, when executed by the processor, cause the processor to function as:
an obtaining unit configured to obtain an image derived from image capture; and
an image processing unit configured to add an effect of a virtual light on a subject included in the image obtained by the obtaining unit, the effect of the virtual light including an effect caused by a specular reflection,
wherein the image processing unit determines the effect caused by the specular reflection based on illuminating condition of the subject by an ambient light.

2. The image processing apparatus according to claim 1 , wherein
the effect of the virtual light further includes an effect caused by a diffuse reflection, and
the image processing unit determines the effect caused by the specular reflection and the effect caused by the diffuse reflection based on the illuminating condition of the subject by the ambient light.

3. The image processing apparatus according to claim 1 , wherein
the instructions further function as a setting unit configured to set an illumination direction of the virtual light, and
the image processing unit further determines the illumination direction of the virtual light based on the illuminating condition of the subject caused by the ambient light.

4. The image processing apparatus according to claim 1 , wherein the illuminating condition of the subject caused by the ambient light is based on a diffusion degree of light emitted by the ambient light.

5. The image processing apparatus according to claim 1 , wherein the illuminating condition of the subject caused by the ambient light is based on a contrast in a subject region of the obtained image.

6. The image processing apparatus according to claim 1 , wherein the illuminating condition of the subject caused by the ambient light is based on a luminance distribution in a subject region of the obtained image.

7. The image processing apparatus according to claim 1 , wherein the illuminating condition of the subject caused by the ambient light is an illumination direction of the ambient light.

8. The image processing apparatus according to claim 1 , wherein the illuminating condition of the subject caused by the ambient light includes at least one of oblique light, follow light and back light.

9. The image processing apparatus according to claim 1 , wherein the image processing unit adds effects of a plurality of virtual lights based on the illuminating condition of the subject caused by the ambient light.

10. The image processing apparatus according to claim 9 , wherein the effect caused by the specular reflection of each of the plurality of virtual lights is defined for a respective virtual light.

11. The image processing apparatus according to claim 9 , wherein the instructions further function as a setting unit configured to respectively set an illuminating direction of each of the plurality of the virtual lights.

12. A control method for an image processing apparatus, the control method comprising:
obtaining an image derived from image capture; and
adding an effect of a virtual light on a subject included in the obtained image, the effect of the virtual light including an effect caused by a specular reflection, wherein
the effect caused by the specular reflection is determined based on illuminating condition of the subject by an ambient light.

13. A non-transitory computer readable storage medium storing a program to be used by an image processing apparatus, the program being configured to cause a computer of the image processing apparatus to execute:
obtaining an image derived from image capture; and
adding an effect of a virtual light on a subject included in the obtained image, the effect of the virtual light including an effect caused by a specular reflection, wherein
the effect caused by the specular reflection is determined based on illuminating condition of the subject by an ambient light.

Assignees

• Canon Kk

Inventors

• Kitajima Kotaro

Classifications

• H04N23/72

  Combination of two or more compensation controls · CPC title

• H04N5/2621 Primary

  Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability · CPC title

• H04N23/74

  by influencing the scene brightness using illuminating means · CPC title

• G06F18/24

  Classification techniques · CPC title

• G06V10/60

  relating to illumination properties, e.g. using a reflectance or lighting model · CPC title