Augmented reality beauty product tutorials

Patent metadata

Field	Value
Publication number	US-12226001-B2
Application number	US-202418609698-A
Country	US
Kind code	B2
Filing date	Mar 19, 2024
Priority date	Mar 31, 2020
Publication date	Feb 18, 2025
Grant date	Feb 18, 2025

Abstract

Systems, methods, and computer readable media for augmented reality beauty product tutorials are disclosed. Methods disclose displaying on a first portion of a screen a beauty product tutorial including pre-recorded images of a presenter applying a beauty product to a body part of the presenter and determining first effects indicating changes to make to live images of a user to simulate applying the beauty product to the body part of the user, where the first effects are determined based on second effects indicating changes to the pre-recorded images from applying the beauty product to the body part of the presenter. Methods further disclose processing the live images of the user to add the first effects and indications of motion of the beauty product and displaying the processed live images of the user on a second portion of the screen.

First claim

What is claimed is:

1. A method comprising:
determining one or more effects, the one or more effects indicating changes to make to live images of a first person to simulate applying a beauty product to a body part of the first person, the one or more effects being determined based on changes to a pre-recorded video depicting the beauty product being applied to the body part of a second person;
processing the live images of the first person to add the one or more effects;
displaying on a screen of a computing device the processed live images of the first person;
displaying options to customize the beauty product; and
in response to a selection of an option of the options to customize the beauty product, processing the processed live images of the first person to modify the one or more effects based on the option to customize the beauty product, and displaying on the screen of the computing device the modified processed live images of the first person.

2. The method of claim 1 , further comprising:
determining one or more indications of motion of the beauty product.

3. The method of claim 2 , further comprising:
processing the live images of the first person to add the one or more indications of motion of the beauty product.

4. The method of claim 3 wherein the one or more indications of motion of the beauty product are determined based on motion of the beauty product within the pre-recorded video.

5. The method of claim 2 wherein an indication of motion of the one or more indications of motion of the beauty product are determined based on comparing a first effect of the one or more effects with a second effect of the one or more effects.

6. The method of claim 1 wherein the displaying further comprises:
displaying, on a first portion of the screen of a computing device, the pre-recorded video and on a second portion of the screen the modified processed live images of the first person.

7. The method of claim 6 further comprising:
determining within the live images of the first person a body part of the first person corresponding to the body part of the second person; and
adjusting the second portion of the screen to display the body part of the first person in a size proportional to a size of the body part of the second person displayed in the pre-recorded images.

8. The method of claim 7 , wherein the options to customize the beauty product are determined based on the body part.

9. The method of claim 6 wherein the pre-recorded video and the modified processed live images are displayed so that a first effect of the one or more effects determined based on changes to the pre-recorded video is displayed simultaneously with a second effect added to the first person, the first effect corresponding to the second effect.

10. The method of claim 1 , wherein the method is performed by an apparatus of the computing device.

11. The method of claim 1 , further comprising:
receiving the live images from a camera of the computing device.

12. The method of claim 1 , further comprising:
displaying an image of the beauty product on the screen; and
in response to a selection of the image of the beauty product, displaying additional information regarding the beauty product, the additional information regarding the beauty product comprising an icon to customize the beauty product.

13. The method of claim 1 , wherein the options to customize the beauty product comprises one or more of: changing a color of the beauty product, changing a brush size of the beauty product, changing an application size of the beauty product, or changing an application thickness of a beauty product.

14. The method of claim 1 , further comprising:
capturing a current live image of the modified processed live images of the first person, the current live image comprising the one or more effects;
displaying the captured current live image on the screen; and
sending the captured current live image as an ephemeral message of a social network.

15. The method of claim 1 , further comprising:
in response to an indication to turn off augmented reality,
processing second live images of the first person to add the one or more indications of motion of the beauty product; and
displaying on the screen the processed second live images of the first person.

16. A system comprising:
one or more processors; and
one or memories including instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
determining one or more effects, the one or more effects indicating changes to make to live images of a first person to simulate applying a beauty product to a body part of the first person, the one or more effects being determined based on changes to a pre-recorded video depicting the beauty product being applied to the body part of a second person;
processing the live images of the first person to add the one or more effects;
displaying on a screen of a computing device the processed live images of the first person;
displaying options to customize the beauty product; and
in response to a selection of an option of the options to customize the beauty product, processing the processed live images of the first person to modify the one or more effects based on the option to customize the beauty product, and displaying on the screen of the computing device the modified processed live images of the first person.

17. The system of claim 16 , wherein the operations further comprise:
determining one or more indications of motion of the beauty product.

18. The system of claim 17 , wherein the operations further comprise:
processing the live images of the first person to add the one or more indications of motion of the beauty product.

19. A non-transitory computer-readable medium comprising instructions, which when executed by a computing device, cause the computing device to perform operations comprising:
determining one or more effects, the one or more effects indicating changes to make to live images of a first person to simulate applying a beauty product to a body part of the first person, the one or more effects being determined based on changes to a pre-recorded video depicting the beauty product being applied to the body part of a second person;
processing the live images of the first person to add the one or more effects;
displaying on a screen of a computing device the processed live images of the first person;
displaying options to customize the beauty product; and
in response to a selection of an option of the options to customize the beauty product, processing the processed live images of the first person to modify the one or more effects based on the option to customize the beauty product, and displaying on the screen of the computing device the modified processed live images of the first person.

20. The non-transitory computer-readable medium of claim 19 , wherein the operations further comprise:
determining one or more indications of motion of the beauty product; and
processing the live images of the first person to add the one or more indications of motion of the beauty product.

Assignees

• Snap Inc

Inventors

• Barron Christine
• Drummond Virginia
• Luo Jean
• Matthiessen Alek
• Mourkogiannis Celia Nicole
• Solichin Jonathan
• Voronova Olesia

Classifications

• G06T11/10

  Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) · CPC title

• G09B19/003

  Repetitive work cycles; Sequence of movements · CPC title

• G06T2200/24

  involving graphical user interfaces [GUIs] · CPC title

• G06Q30/0601

  Electronic shopping [e-shopping] · CPC title

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title