Adding beauty products to augmented reality tutorials

Patent metadata

Field	Value
Publication number	US-12046037-B2
Application number	US-202318232680-A
Country	US
Kind code	B2
Filing date	Aug 10, 2023
Priority date	Jun 10, 2020
Publication date	Jul 23, 2024
Grant date	Jul 23, 2024

Abstract

Systems, methods, and computer-readable media for adding beauty products to tutorials are presented. Methods include accessing video data comprising images of a presenter creating a tutorial, the tutorial depicting the presenter applying a beauty product to a body part of the presenter. Methods further include processing the video data to identify changes to the body part of the presenter from an application of the beauty product, and responding to identifying changes to the body part of the presenter from the application of the beauty product by processing the video data to identify the beauty product. Methods further include retrieving information regarding the beauty product and causing presentation of information regarding the beauty product on a display device.

First claim

What is claimed is:

1. A system comprising:
one or more processors; and
one or more memories including instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
identifying a beauty product to be applied to a body part;
accessing video data comprising images of a presenter creating a tutorial, the tutorial depicting the beauty product being applied to the body part of a person;
identifying, based on at least a color of the beauty product, changes to the body part of the person from an application of the beauty product; and
associating the changes to the body part of the person with the beauty product.

2. The system of claim 1 , wherein the operations further comprise:
causing information regarding the beauty product to be displayed on a display device.

3. The system of claim 1 , wherein the operations further comprise:
storing, in a datastore associated with the system, association data, the association data indicating the changes to the body part of the person with the beauty product.

4. The system of claim 1 , wherein the person is a presenter of the tutorial.

5. The system of claim 1 , wherein the changes to the body part of the person comprise a color change of the body part or a change of texture of the body part.

6. The system of claim 1 , wherein the operations further comprise:
retrieving information regarding the beauty product, the information comprising the color of the beauty product.

7. The system of claim 6 , wherein the information regarding the beauty product further comprises one or more of: a name of the beauty product, the body part associated with the beauty product, a manufacturer of the beauty product, a website associated with the beauty product, an image of the beauty product, a product code of the beauty product, and an indication of ratings of the beauty product.

8. The system of claim 1 , wherein the video data is live video data, the live video data captured by a camera associated with the system.

9. The system of claim 1 , wherein the operations further comprise:
determining effects indicating changes to make to second video data of a user to simulate applying the beauty product to the body part of the user, wherein the effects are determined based on the changes to the body part of the presenter; and
storing, within a data store, association data associating the effects with the beauty product and the tutorial.

10. The system of claim 9 , wherein the changes to the body part of the person comprise one or more of: a color, a texture, a transparency, a width, a length, a skin type for the body part, and a hair type of the body part.

11. The system of claim 1 , wherein identifying the beauty product to be applied to the body part further comprises:
receiving input from the person that identifies the beauty product.

12. The system of claim 1 , wherein the operations further comprise:
recording the video data using a computing device;
during the recording of the video data, receiving input from the person that identifies the beauty product applied to the body part; and
focusing a display, on the computing device, of the video data on images of the beauty product.

13. The system of claim 1 , wherein the operations further comprise:
identifying a product code from an image of the beauty product;
matching the product code to stored product codes to identify the beauty product; and
retrieving information regarding the beauty product using the product code, wherein the product code is an identifier assigned to the beauty product.

14. The system of claim 13 , wherein the information regarding the beauty product comprises an identity of the beauty product, and wherein the operations further comprise:
causing presentation of a user interface item for the person to confirm or reject the identity of the beauty product; and
in response to receiving, via an application implemented by one or more processors, input from the person that indicates rejection of the identity of the beauty product, recording new images of the person creating the tutorial, and identifying the beauty product based on the new images.

15. The system of claim 1 , wherein the video data is live video data, and wherein the operations further comprise:
in response to determining that the beauty product is within the live video data,
causing an interrogation signal to be transmitted to the beauty product to solicit a radio frequency identification (RFID) identifying the beauty product,
receiving radio signals comprising the RFID from the beauty product, the RFID comprising a product code, and
identifying the beauty product using the product code.

16. A non-transitory computer-readable medium comprising instructions, which when executed by a computing device comprising one or more processors, cause the one or more processors to:
identifying a beauty product to be applied to a body part;
accessing video data comprising images of a presenter creating a tutorial, the tutorial depicting the beauty product being applied to the body part of a person;
identifying, based on at least a color of the beauty product, changes to the body part of the person from an application of the beauty product; and
associating the changes to the body part of the person with the beauty product.

17. The non-transitory computer-readable medium of claim 16 , wherein the instructions further comprise:
causing information regarding the beauty product to be displayed on a display device.

18. The non-transitory computer-readable medium of claim 16 , wherein the instructions further comprise:
storing, in a datastore associated with the computing device, association data, the association data indicating the changes to the body part of the person with the beauty product.

19. A method, comprising:
identifying a beauty product to be applied to a body part;
accessing video data comprising images of a presenter creating a tutorial, the tutorial depicting the beauty product being applied to the body part of a person;
identifying, based on at least a color of the beauty product, changes to the body part of the person from an application of the beauty product; and
associating the changes to the body part of the person with the beauty product.

20. The method of claim 19 further comprising:
causing information regarding the beauty product to be displayed on a display device.

Assignees

• Snap Inc

Inventors

• Barron Christine
• Drummond Virginia
• Luo Jean
• Matthiessen Alek
• Mourkogiannis Celia Nicole
• Solichin Jonathan
• Voronova Olesia

Classifications

• G06V20/46

  Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames · CPC title

• G06Q30/0627 Primary

  by specifying product or service characteristics, e.g. product dimensions · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06K7/10118

  the sensing being preceded by at least one preliminary step · CPC title

• A61Q1/00

  Make-up preparations; Body powders; Preparations for removing make-up · CPC title