User interface for pose driven virtual effects

Patent metadata

Field	Value
Publication number	US-12452384-B2
Application number	US-202318520255-A
Country	US
Kind code	B2
Filing date	Nov 27, 2023
Priority date	Aug 13, 2020
Publication date	Oct 21, 2025
Grant date	Oct 21, 2025

Abstract

Systems and methods herein describe a method for capturing a video in real-time by an image capture device. The system provides a plurality of visual pose hints, identifies first pose information in the video while capturing the video, applies a first series of virtual effects to the video, identifies second pose information, and applies a second series of virtual effects to the video, the second series of virtual effects based on the first series of virtual effects.

First claim

What is claimed is:

1. A method comprising:
causing display of a skeletal pose tracking system on a graphical user interface of a computing device, the graphical user interface comprising an image of a human body;
receiving, from the computing device, a selection of one or more regions of the human body;
receiving augmented reality effect data for each of the one or more regions of the human body, the augmented reality effect data comprising a virtual effect, a trigger condition and an accuracy threshold; and
causing modification of a video comprising a human user based on the received augmented reality effect data.

2. The method of claim 1 , wherein the video comprises first pose information representing a first plurality of joint locations of the human user depicted in the video.

3. The method of claim 1 , wherein the video comprises second pose information representing a second plurality of joint locations of the human user depicted in the video.

4. The method of claim 1 , wherein the video comprises a first series of virtual effects comprising a plurality of first augmented reality content item.

5. The method of claim 4 , wherein the plurality of first augmented reality content items is applied to the video in real-time during capture.

6. The method of claim 4 , wherein the video comprises a second series of virtual effects comprising a plurality of second augmented reality content items.

7. The method of claim 6 , further comprising:
storing the video comprising the first series of virtual effects at a first time and the second series of virtual effects at a second time; and
transmitting the video as an ephemeral message to a second computing device.

8. The method of claim 1 , further comprising:
identifying a hand in the video;
tracking a motion of the hand from a first position to a second position; and
modifying a level of granularity of the virtual effect based on the tracked motion of the hand.

9. A system comprising:
a processor; and
a memory storing instructions that, when executed by the processor, configure the system to perform operations comprising:
causing display of a skeletal pose tracking system on a graphical user interface of a computing device, the graphical user interface comprising an image of a human body;
receiving, from the computing device, a selection of one or more regions of the human body;
receiving augmented reality effect data for each of the one or more regions of the human body, the augmented reality effect data comprising a virtual effect, a trigger condition and an accuracy threshold; and
causing modification of a video comprising a human user based on the received augmented reality effect data.

10. The system of claim 9 , wherein the video comprises first pose information representing a first plurality of joint locations of the human user depicted in the video.

11. The system of claim 9 , wherein the video comprises second pose information representing a second plurality of joint locations of the human user depicted in the video.

12. The system of claim 9 , wherein the video comprises a first series of virtual effects comprising a plurality of first augmented reality content item.

13. The system of claim 12 , wherein the plurality of first augmented reality content items are applied to the video in real-time during capture.

14. The system of claim 12 , wherein the video comprises a second series of virtual effects comprising a plurality of second augmented reality content items.

15. The system of claim 14 , wherein the operations further comprise:
storing the video comprising the first series of virtual effects at a first time and the second series of virtual effects at a second time; and
transmitting the video as an ephemeral message to a computing device.

16. The system of claim 9 , further comprising:
identifying a hand in the video;
tracking a motion of the hand from a first position to a second position; and
modifying a level of granularity of the virtual effect based on the tracked motion of the hand.

17. A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to perform operations comprising:
causing display of a skeletal pose tracking system on a graphical user interface of a computing device, the graphical user interface comprising an image of a human body;
receiving, from the computing device, a selection of one or more regions of the human body;
receiving augmented reality effect data for each of the one or more regions of the human body, the augmented reality effect data comprising a virtual effect, a trigger condition and an accuracy threshold; and
causing modification of a video comprising a human user based on the received augmented reality effect data.

18. The computer-readable storage medium of claim 17 , wherein the video comprises first pose information representing a first plurality of joint locations of the human user depicted in the video.

19. The computer-readable storage medium of claim 17 , wherein the video comprises second pose information representing a second plurality of joint locations of the human user depicted in the video.

20. The computer-readable storage medium of claim 17 , wherein the video comprises a first series of virtual effects comprising a plurality of first augmented reality content item.

Assignees

• Snap Inc

Inventors

• Alavi Amir
• Rykhliuk Olha
• Shi Xintong
• Solichin Jonathan
• Voronova Olesia
• Yagodin Artem

Classifications

• H04N23/631

  Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters · CPC title

• H04N23/611

  where the recognised objects include parts of the human body · CPC title

• G06T2207/30196

  Human being; Person · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T11/00

  Two-dimensional [2D] image generation · CPC title