Systems and methods for specifying and formulating customized topical agents

Patent metadata

Field	Value
Publication number	US-11445803-B2
Application number	US-201916427111-A
Country	US
Kind code	B2
Filing date	May 30, 2019
Priority date	Mar 15, 2013
Publication date	Sep 20, 2022
Grant date	Sep 20, 2022

Abstract

Systems and methods for determining a customized cosmetic formulation. In one method, a user is guided to capture an image of a skin region with known lighting and color characteristics, and the image is processed to provide calibrated skin color information. A customized cosmetic formulation is automatically determined for the user based on the calibrated skin color information. In another method, a user is interactively guided through capture of one or more skin region images using a device having an image sensor. The skin region images are processed to provide calibrated skin color information, which is compared to a ground truth data set to identify a set of closest known values in the ground truth data set. A customized cosmetic formulation is automatically determined for the user based on the comparison.

First claim

What is claimed is:

1. A method of determining a customized cosmetic formulation, the method comprising:
initiating a software application on a device having an image sensor and a light source;
actuating the software application on the device, wherein the software application
prompts a user to capture at least one image comprising a skin region of a user and
controls the image sensor of the device for determining an amount of ambient light and setting the light source;
guides the user to capture the at least one image comprising the skin region of the user when a distance to the user's skin region is within a set distance and when the user's skin region is centered and in focus so as to meet a calibration target for the device having the image sensor, wherein the calibration target is associated with preferred settings for the device having the image sensor, including exposure, focus and white balance settings;
processes the at least one captured image using embedded image data from the at least one captured image, including brightness and flash characteristics of the device, focal length data from the light source and using known characteristics of the skin region in the at least one captured image and alters the device settings, if needed, to be the preferred settings for the device based on the embedded image data to calibrate the device;
locks the exposure, focus and white balance settings on the device after calibrating the device having the image sensor;
and guides the user to take one or more additional images of the skin region of the user using the calibrated device having the image sensor;

processing the one or more additional images of the skin region from the calibrated device to provide calibrated skin color information for the user, including user device data; and
determining a custom cosmetic formulation for the user using the calibrated skin color information by matching the calibrated skin color information to reference skin color information of a reference user in a reference database, the reference user having a previously determined cosmetic formulation associated therewith.

2. The method of claim 1 , further comprising guiding the user to capture of a plurality of skin region images, which images may be the same or a different skin region.

3. The method of claim 1 , further comprising guiding the user through capture of an object using the device image sensor, wherein the skin color information is also calibrated by processing the captured image of the object.

4. The method of claim 3 , wherein the object having known characteristics comprises at least one of a marker and a further captured image of a skin region of the user having known characteristics, which may be the same or a different skin region.

5. The method of claim 3 , wherein the object comprises a color reference chart.

6. The method of claim 5 , wherein the color reference chart is displayed on a screen of the device and the user captures the image of the skin region in a mirror including the displayed color reference chart.

7. The method of claim 3 , wherein the known characteristics of the object comprise known size and/or known reflectance.

8. The method of claim 1 , wherein the device is selected from the group consisting of a mobile phone, a smart phone, a tablet, a digital camera, a webcam, smart glasses, a television, and a gaming console.

9. The method of claim 1 , wherein the previously determined cosmetic formulation associated with the reference user is a non-custom product.

10. The method of claim 1 , wherein the determining the custom cosmetic formulation is based at least in part on an indication by the user of an allergy to a cosmetic ingredient.

11. The method of claim 1 , further comprising compounding a topical agent based on the custom cosmetic formulation.

12. The method of claim 11 , further comprising delivering the topical agent to the user.

13. The method of claim 1 , further comprising allowing the user to purchase a topical agent compounded based on the custom cosmetic formulation.

14. The method of claim 1 , wherein the skin region comprises a facial region of the user.

15. The method of claim 1 , wherein the device is a smart phone and the user generates the captured image by taking the image using the image sensor of the smart phone in a display screen of the smart phone.

16. The method of claim 1 , wherein the device is a smart phone having a flash and/or a torch and the flash and/or the torch may be used as the light source for capturing the image of the skin region.

17. The method according to claim 1 , wherein the device is a smart phone comprising a light source and the actuating the software application to control the image sensor of the device further comprises activating a facial detection software on the device.

18. The method according to claim 1 , wherein determining the custom cosmetic formulation further comprises comparing the calibrated skin color information to a ground truth data set to identify a set of closest known values in the ground truth data set.

19. The method according to claim 1 , wherein the method further comprises processing the captured image, at least in part, based on ambient lighting information received from the calibration of the device having the image sensor.

20. A system for determining a customized cosmetic formulation, the system comprising:
at least one computer comprising a memory for storing computer-executable instructions and a processing unit for executing the instructions stored in the memory, wherein the execution of the instructions configures the computer to perform operations comprising:
initiating a software application on a device having an image sensor and a light source;
actuating the software application on the device, wherein the software application
prompts a user to capture at least one image comprising a skin region of a user;
controls the image sensor of the device for determining an amount of ambient light and setting the light source;
guides the user to capture the at least one image comprising the skin region of the user when a distance to the user's skin region is within a set distance and when the user's skin region is centered and in focus so as to meet a calibration target for the device having the image sensor, wherein the calibration target is associated with preferred settings for the device having the image sensor, including exposure, focus and white balance settings;
processes the at least one captured image using embedded image data from the at least one captured image, including brightness and flash characteristics of the device, focal length data from the light source and using known characteristics of the skin region in the at least one captured image and alters the device settings, if needed, to be the preferred settings for the device based on the embedded image data to calibrate the device;
locks the exposure, focus and white balance settings on the device after calibrating the device having the image sensor;
and guides the user to take one or more additional images of the skin region of the user using the calibrated device having the image sensor;

processing the one or more additional images of the skin region from the calibrated device to provide calibrated skin color information for the user, including user device data; and
determining a custom cosmetic formulation for the user using the calibrated skin color information by matching the calibrated skin color information to reference skin color information

Assignees

• Shiseido Co Ltd

Inventors

• Howell David A
• Gross David B
• Harvill Leslie Y

Classifications

• G06V40/162

  using pixel segmentation or colour matching · CPC title

• A45D44/005 Primary

  for selecting or displaying personal cosmetic colours or hairstyle · CPC title

• A61B5/1032

  Determining colour of tissue for diagnostic purposes · CPC title

• G01J3/463

  Colour matching · CPC title

• A61B5/441

  Skin evaluation, e.g. for skin disorder diagnosis · CPC title