Systems and methods for determining apparent skin age

Patent metadata

Field	Value
Publication number	US-10818007-B2
Application number	US-201815993950-A
Country	US
Kind code	B2
Filing date	May 31, 2018
Priority date	May 31, 2017
Publication date	Oct 27, 2020
Grant date	Oct 27, 2020

Abstract

Systems and methods for determining an apparent skin age of a person and providing customized skin care product recommendations. The system utilizes a computing device to mask facial macro features in an image of a person and then analyze the image with a convolutional neural network to determine an apparent skin age of the person. Determining the apparent skin age may include identifying at least one pixel that is indicative of skin age and utilizing the at least one pixel to provide the apparent skin age. The system may be used to determine a target skin age of a person, determine a skin care product or skin care regimen for achieving the target skin age, and provide an option for a user to purchase the product.

First claim

What is claimed is:

1. A system for determining an apparent skin age of a person, comprising: a non-transitory computer readable storage medium with logic stored thereon, wherein the logic causes the system to
a) receive a digital image comprising a human face;
b) process the digital image for analysis, wherein processing comprises locating the human face in the digital image, segmenting the digital image into two or more image segments, and masking at least one macro feature present on the face;
c) analyze the processed image using a convolutional neural network (CNN) comprising a discrete deep neural network for each image segment, wherein the analysis includes identifying in a region of interest in each image segment at least one pixel disposed in a facial micro feature that is indicative of the person's skin age;
d) determine with the CNN an overall apparent skin age of the person based on the analysis of each deep neural network; and
e) display the overall apparent skin age on a display device visible to a user.

2. The system of claim 1 , further comprising an image capture device coupled to a computer, wherein the digital image is captured by the image capture device and received by the computer.

3. The system of claim 1 , wherein masking the macro feature comprises replacing pixels in the facial macro feature with pixels that have a median RGB value of skin disposed in a region of interest.

4. The system of claim 1 , wherein a result of each analysis from each deep neural network is used to determine an apparent skin age for that region of interest.

5. The system of claim 4 , wherein results from all the deep neural network analyses are concatenated to provide an overall apparent skin age.

6. The system of claim 1 , wherein processing the image further includes providing two or more input variations to the CNN.

7. The system of claim 6 , wherein the image is segmented and then the macro feature is masked to provide a first input variation.

8. The system of claim 7 , wherein the macro feature is masked and then the image is segmented to provide a second input variation.

9. A method of determining the apparent skin age of a person, comprising:
a) receiving an image of the person, wherein the image includes at least a portion of the person's face;
b) processing the image with a computer, wherein processing the image includes identifying the portion of the image comprising the face, segmenting the digital image into two or more image segments, and masking a macro feature of the face;
c) analyzing the image using a convolutional neural network comprising a discrete deep neural network for each image segment stored on a memory component of the computer to provide an apparent skin age, wherein analyzing the image includes identifying in a region of interest in each image segment at least one pixel that is indicative of skin age and utilizing the at least one pixel from each image segment to provide an overall apparent skin age; and
d) displaying the overall apparent skin age on a display device visible to a user.

10. The method of claim 9 , further comprising an image capture device coupled to a computer, wherein the digital image is captured by the image capture device and received by the computer.

11. The method of claim 9 , wherein masking the macro feature comprises replacing pixels in the facial macro feature with pixels that have a median RGB value of skin disposed in a region of interest.

12. The method of claim 9 , wherein a result of each analysis from each deep neural network is used to determine an apparent skin age for that region of interest.

13. The method of claim 12 , further comprising concatenating results from all the deep neural network analyses to provide the overall apparent skin age.

14. The method of claim 9 , wherein processing the image further includes providing two or more input variations to the CNN.

15. The method of claim 14 , wherein the image is segmented and then the macro feature is masked to provide a first input variation.

16. The method of claim 15 , wherein a first input variation is provided by segmenting the image and then masking the macro feature, and a second input variation is provided by masking the macro feature and then segmenting the image.

17. A system for determining an apparent skin age of a person, comprising: a non-transitory computer readable storage medium with logic stored thereon, wherein the logic causes the system to:
a) receive a digital image comprising a human face;
b) process the digital image for analysis, wherein processing comprises locating the human face in the digital image and masking at least one macro feature present on the face;
c) segmenting the digital image into two or more image segments;
d) scaling the segmented digital image such that the full height of the facial image does not exceed 800 pixels;
e) bounding the digital image in a bounding box to remove at least one of a background feature and a macro feature;
f) analyze the processed image using a convolutional neural network (CNN) comprising a discrete deep neural network for each image segment, wherein each deep neural network is trained to identify at least one pixel disposed in a facial micro feature that is indicative of the person's skin age, and wherein each discrete neural network generates an apparent skin age for a region of interest in its respective image segment;
g) determine with the CNN an overall apparent skin age of the person based on the apparent skin age from each deep neural network; and
h) display the overall apparent skin age on a display device visible to a user.

18. The system of claim 17 , wherein the human face is located in the digital image using a Viola-Jones weak cascade technique.

Assignees

• Procter & Gamble
• Palo Alto Res Ct Inc

Inventors

• Purwar Ankur
• Matts Paul Jonathan
• Shreve Matthew Adam
• Wu Wencheng
• Xu Beilei

Classifications

• G06V10/82

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06V10/273 Primary

  removing elements interfering with the pattern to be recognised · CPC title

• G06T7/0012 Primary

  Biomedical image inspection · CPC title

• G06F18/24143

  Distances to neighbourhood prototypes, e.g. restricted Coulomb energy networks [RCEN] · CPC title