Makeup application assistance device, makeup application assistance method, and makeup application assistance program

Patent metadata

Field	Value
Publication number	US-10242589-B2
Application number	US-201615010200-A
Country	US
Kind code	B2
Filing date	Jan 29, 2016
Priority date	Feb 1, 2013
Publication date	Mar 26, 2019
Grant date	Mar 26, 2019

Abstract

A makeup assisting apparatus includes a first image acquirer that acquires a first image which is a captured image of a first face, a first facial part acquirer that acquires an area of a portion of the first face, a second image acquirer that acquires a second image which is a captured image of a second face to which makeup is applied, and a second facial part acquirer that acquires an area of a portion of the second face. A reference makeup acquirer acquires makeup from the second image for each of the portions of the second face, based on the area of the portion of the second face, the makeup being a way of applying the makeup and a makeup presenter that presents, to a user, the acquired makeup in association with the corresponding area of the portion of the first face.

First claim

What is claimed is:

1. A makeup assisting apparatus including a processor that executes instructions and a memory that stores the instructions and configured to:
acquire a stereo image, which comprises a first image, which is an image obtained by capturing a user's face when the user's face is makeup free and generate a makeup removed image of the user's face from the first image to be used as the first image for subsequent processing when the user's face has makeup thereon, and map, when the first image of the user's face captured from an oblique direction is acquired, the first image to a stereoscopic shape of the user's face to generate a first generated image when the user's face is seen from a front;
acquire areas of a plurality of portions of the user's face from the first generated image;
acquire a second image, which is obtained by capturing a model's face, to which makeup is applied and map, when the second image of the model's face captured from an oblique direction is acquired, the second image to a stereoscopic shape of the model's face, to generate a second generated image when the model's face is seen from the front;
acquire areas of a plurality of portions of the model's face from the second generated image;
specify an area having a color approximate to a color of a skin of the model's face, from a positional relationship between the areas of the plurality of portions of the model's face;
estimate, for each of the plurality of portions of the model's face in the second generated image, skin colors of the plurality of portions of the model's face in the second generated image, based on the color of the area specified;
acquire, from the second generated image, by reverse alpha blending, reference makeup and the color and concentration of the reference makeup including the level of permeability thereof and the gradation thereof for each of the plurality of portions of the model's face, based on differences between the skin colors of the plurality of portions of the second generated image and actual colors of the plurality of portions of the second generated image, the reference makeup including a way of applying the makeup;
acquire the reference makeup from a two-dimensional code displayed on an image of the model's face in a picture thereof instead of acquiring the reference makeup based on differences between the skin colors of the plurality of portions of the second image and actual colors of the plurality of portions of the second image, when acquiring the second image by capturing a picture of the model's face that includes the two-dimensional code; and
transfer, with a printer, a cosmetic agent of skin cosmetics required for applying the reference makeup to a peelable predetermined thin film transferable to the user's face when peeled after the printer is connected to the makeup assisting apparatus.

2. The makeup assisting apparatus according to claim 1 , wherein the processor is further configured to control a display to display a two-dimensional coordinate system in the first image of the user's face in association with a three-dimensional coordinate system in the stereoscopic shape of the user's face, based on feature points of the user's face, to generate the first generated image of the face.

3. A makeup assisting method for a makeup assisting apparatus comprising:
acquiring a stereo image, which comprises a first image obtained by capturing a user's face when the user's face is makeup free and generate a makeup removed image of the user's face from the first image to be used as the first image for subsequent processing when the user's face has makeup thereon;
mapping, when the first image of the user's face captured from an oblique direction is acquired, the first image to a stereoscopic shape of the user's face to generate a first generated image when the user's face is seen from a front;
acquiring areas of a plurality of portions of the user's face from the first generated image;
acquiring a second image which is an image obtained by capturing a model's face to which makeup is applied;
mapping, when the second image of the model's face captured from an oblique direction is acquired, the second image to a stereoscopic shape of the model's face to generate a second generated image when the model's face is seen from the front;
acquiring areas of a plurality of portions of the model's face from the second generated image;
specifying an area having a color approximate to a color of a skin of the model's face, from a positional relationship between the areas of the plurality of portions of the model's face;
estimating, for each of the plurality of portions of the model's face in the second generated image, skin colors of the plurality of portion of the model's face in the second generated image, based on the color of the area specified;
acquiring, from the second generated image, by reverse alpha blending, reference makeup and the color and concentration of the reference makeup including the level of permeability thereof and the gradation thereof for each of the plurality of portions of the model's face, based on differences between the skin colors of the plurality of portions of the second generated image and actual colors of the plurality of portions of the second generated image, the reference makeup including a way of applying the makeup;
acquiring the reference makeup from a two-dimensional code displayed on an image of the model's face in a picture thereof instead of acquiring the reference makeup based on differences between the skin colors of the plurality of portions of the second image and actual colors of the plurality of portions of the second image, when acquiring the second image by capturing a picture of the model's face that includes the two-dimensional code;
transferring, with a printer, a cosmetic agent of skin cosmetics required for applying the reference makeup to a peelable predetermined thin film transferable to the user's face when peeled after the printer is connected to the makeup assisting apparatus.

4. The makeup assisting method according to claim 3 , further comprising:
associating a two-dimensional coordinate system in the image of the user's face with a three-dimensional coordinate system in the stereoscopic shape of the user's face, based on feature points of the user's face, to generate the first generated image of the face; and
displaying the first generated image.

5. A non-transitory computer-readable storage medium comprising a make-up assisting program for a make-up assisting apparatus that causes a computer to execute processing comprising:
acquiring a stereo image, which comprises a first image obtained by capturing a user's face when the user's face is makeup free and generate a makeup removed image of the user's face from the first image to be used as the first image for subsequent processing when the user's face has makeup thereon;
mapping, when the first image of the user's face captured from an oblique direction is acquired, the first image to a stereoscopic shape of the user's face to generate a first generated image when the user's face is seen from a front;
acquiring areas of a plurality of portions of the user's face from the first generated image;
acquiring a second image which is an image obtained by capturing a model's face to which makeup is applied;
mapping, when the second image of the model's face captured from an oblique direction is acquired, the second image to a stereoscopic shape of the model's face to generate a second generated image when the model's face is seen from the front;
acquiring areas of a plurality of portions of the model's face from the second generated image;
specifying an area having a color approximate to a color of a skin of the model's face, from a positional relationship between the areas of plur

Assignees

• Panasonic Ip Man Co Ltd

Inventors

• Yamanashi Tomofumi
• Asai Rieko
• Muta Aoi
• Nishi Chie
• Ajiki Kaori

Classifications

• G06T11/00

  Two-dimensional [2D] image generation · CPC title

• A45D44/005

  for selecting or displaying personal cosmetic colours or hairstyle · CPC title

• G06F40/134

  Hyperlinking · CPC title

• G09B19/00 Primary

  Teaching not covered by other main groups of this subclass (teaching or practice apparatus for gun-aiming or gun-laying F41G3/26) · CPC title

• G06T11/60

  Creating or editing images; Combining images with text · CPC title