Apparatus and method for adjusting brightness of image

Patent metadata

Field	Value
Publication number	US-9922452-B2
Application number	US-201615250980-A
Country	US
Kind code	B2
Filing date	Aug 30, 2016
Priority date	Sep 17, 2015
Publication date	Mar 20, 2018
Grant date	Mar 20, 2018

Abstract

A method of adjusting a brightness of an image includes matching an object model to an object based on one or more feature points of the object extracted from an input image including the object; mapping a surface normal map in a two-dimensional (2D) image form to the input image based on the matched object model; and generating shadow information for the input image based on the mapped surface normal map and a virtual light source.

First claim

What is claimed is:

1. A method of adjusting a brightness of an image, the method comprising:
matching an object model to an object based on one or more feature points of the object extracted from an input image comprising the object;
generating a surface normal map by combining at least two of surface normal models corresponding to different views, the combining being based on results of matching feature points of each of the at least two of the surface normal models to feature points of the matched object model; and
generating shadow information for the input image based on the surface normal map and a virtual light source.

2. The method of claim 1 , wherein the generating of the shadow information comprises performing a dot product calculation between a normal vector at each coordinate of the surface normal map and a light vector of the virtual light source at a corresponding coordinate of the input image; and
the performing of the dot product calculation comprises determining the light vector at the corresponding coordinate of the input image based on either a relationship between a location of the virtual light source and the corresponding coordinate of the input image, or an illumination direction of the virtual light source.

3. The method of claim 1 , further comprising:
generating a relighted image by applying the shadow information to the input image; and
adjusting a tone of the relighted image based on a brightness histogram of the input image;
wherein the adjusting of the tone of the relighted image comprises adding a weight to a tone of a pixel in at least a portion of the relighted image based on a ratio of a number of pixels in a preset brightness range in the input image to a total number of pixels in the input image.

4. The method of claim 1 , further comprising receiving a control signal indicating any one or any combination of an intensity, a location, a color, an illumination direction, and a type of the virtual light source;
wherein the object is a human face.

5. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform the method of claim 1 .

6. The method of claim 1 , wherein the method is performed in a smartphone comprising a screen, and further comprises:
displaying on the screen a graphical user interface configured to enable a user of the smartphone to set an intensity of the virtual light source and either a location of the virtual light source or an illumination direction of the virtual light source; and
setting the intensity of the virtual light source and either the location of the virtual light source or the illumination direction of the virtual light source in response to manipulation of the graphical user interface by the user.

7. An apparatus for adjusting a brightness of an image, the apparatus comprising:
a processor configured to:
match an object model to an object based on one or more feature points of the object extracted from an input image comprising the object,
generate a surface normal map by combining at least two of surface normal models corresponding to different views, the combining being based on results of matching feature points of each of the at least two of the surface normal models to feature points of the matched object model, and
generate shadow information for the input image based on the surface normal map and a virtual light source.

8. The apparatus of claim 7 , wherein the processor is further configured to:
perform a dot product calculation between a normal vector at each coordinate of the surface normal map and a light vector of the virtual light source at a corresponding coordinate of the input image, and
determine the light vector at the corresponding coordinate of the input image based on either a relationship between a location of the virtual light source and the corresponding coordinate of the input image, or an illumination direction of the virtual light source.

9. The apparatus of claim 7 , wherein the processor is further configured to:
generate a relighted image by applying the shadow information to the input image, and adjust a tone of the relighted image based on a brightness histogram of the input image, and
add a weight to a tone of a pixel in at least a portion of the relighted image based on a ratio of a number of pixels in a preset brightness range in the input image to a total number of pixels in the input image.

10. The apparatus of claim 7 , wherein the apparatus is a smartphone comprising a screen; and
the processor is further configured to display on the screen a graphical user interface configured to enable a user of the smartphone to set an intensity of the virtual light source and either a location of the virtual light source or an illumination direction of the virtual light source.

11. An apparatus comprising:
an image receiver configured to receive an input image comprising an object; and
a processor configured to:
match an object model to the object based on one or more feature points of the object extracted from the input image,
generate a surface normal map by combining at least two of surface normal models corresponding to different views, the combining being based on results of matching feature points of each of the at least two of the surface normal models to feature points of the matched object model, and
generate shadow information for the input image based on the surface normal map and a virtual light source.

12. The apparatus of claim 11 , wherein the image receiver is a camera;
the processor is further configured to apply the shadow information to the input image; and
the apparatus further comprises a display configured to display the input image to which the shadow information is applied.

13. The apparatus of claim 11 , wherein the processor is further configured to generate the shadow information based on a view vector indicating a direction in which a user views the object and a reflection vector indicating a direction in which light projected from the virtual light source onto the object is reflected by the object.

14. The apparatus of claim 11 , wherein the apparatus is a camera configured to capture the input image and comprising a screen, or a communicator configured to receive the input image from an external source through wired or wireless communication and comprising a screen; and
the processor is further configured to display on the screen a graphical user interface configured to enable a user of the camera or communicator to set an intensity of the virtual light source and either a location of the virtual light source or an illumination direction of the virtual light source.

15. A method of adjusting a brightness of an image, the method comprising:
generating a surface normal map for an object in an input image by combining at least two of surface normal models corresponding to different views, the combining being based on results of matching feature points of each of the at least two of the surface normal models to feature points of an object model matched to the object;
generating shadow information for the input image based on the surface normal map and a virtual light source; and
applying the shadow information to the input image to adjust a brightness of the input image.

16. The method of claim 15 , further comprising matching the object model to the object based on feature points of the object and the feature points of the object model.

17. The method of claim 15 , wherein the generating of the shadow information comprises calculating, as the shado

Assignees

• Samsung Electronics Co Ltd

Inventors

• Son Minjung
• Lee Heesae
• Sagong Donghoon
• Lee Keechang
• Lee Hyong Euk

Classifications

• G06T15/50

  Lighting effects · CPC title

• G06T15/80

  Shading · CPC title

• G06T15/506 Primary

  Illumination models · CPC title

• G06T2207/10004

  Still image; Photographic image · CPC title

• G06T15/60 Primary

  Shadow generation · CPC title