Triggering changes to real-time special effects included in a live streaming video

Patent metadata

Field	Value
Publication number	US-11212331-B1
Application number	US-201916264515-A
Country	US
Kind code	B1
Filing date	Jan 31, 2019
Priority date	Oct 26, 2018
Publication date	Dec 28, 2021
Grant date	Dec 28, 2021

Abstract

Method for triggering changes to real-time special effects included in a live streaming video starts with a processor transmitting in real-time a video stream captured by a camera via a network. The processor causes a live streaming interface that includes the video stream to be displayed on the plurality of client devices. The processor receives a trigger to apply one of a plurality of special effects to the video stream and determines a first special effect of the plurality of special effects is associated with the trigger. The processor applies in real-time the first special effect to the video stream to generate a video stream having the first special effect and transmits in real-time the video stream having the first special effect via the network. The processor causes the live streaming interface that includes the video stream having the first special effect to be displayed on the plurality of client devices. Other embodiments are disclosed.

First claim

What is claimed is:

1. A system of triggering changes to real-time special effects included in a live streaming video comprising:
a camera to capture a video stream of a user that is transmitted in real-time to a plurality of client devices, the video stream of the user comprising images of the user;
a display device;
a processor; and
a memory component having instructions stored thereon, when executed by the processor, causes the system to perform operations comprising:
transmitting in real-time the video stream,
causing a live streaming interface to be displayed on the plurality of client devices, wherein the live streaming interface includes the video stream and a live stream of the video game being played by the user in real-time, wherein the video stream and the live stream of a video game are displayed independently in the live streaming interface,
receiving a trigger to apply one of a plurality of special effects to the video stream, wherein each of the plurality of special effects is an augmented reality special effect using a combination of a visual effect applied to the user's face in the video stream and a sound effect applied to audio of the video stream,
determining a first special effect of the plurality of special effects is associated with the trigger,
applying in real-time the first special effect to the video stream to generate a video stream having the first special effect,
transmitting in real-time the video stream having the first special effect via the network, and
causing the live streaming interface including the video stream having the first special effect to be displayed on the plurality of client devices.

2. The system of claim 1 , wherein receiving the trigger includes:
receiving a selection of a display item included in the live streaming interface from one of the plurality of client devices.

3. The system of claim 2 , wherein determining the first special effect is associated with the trigger includes:
determining that the first special effect is associated with the display item.

4. The system of claim 3 , wherein the system to perform operations further comprising:
receiving a customized setting that associates the display item with the first special effect.

5. The system of claim 1 , wherein the system to perform operations further comprising:
causing a lens selection user interface to be displayed on the display device, the lens selection user interface including a plurality of lens selections,
wherein the plurality of lens selections are associated with a plurality of special effects, respectively,
wherein the plurality of lens selections include a first lens selection associated with the first special effect and a second lens selection associated with a second special effect.

6. The system of claim 5 , wherein receiving the trigger includes:
receiving an input from a user via the lens selection user interface selecting the first lens selection, or
receiving an input from the user of a set of keys on a keyboard, wherein the set of keys is associated with the first lens selection.

7. The system of claim 1 , wherein the system to perform operations further comprising:
receiving a subsequent trigger to apply one of a plurality of special effects to the video stream,
determining a second special effect of the plurality of special effects is associated with the subsequent trigger,
applying in real-time the second special effect to the video stream to generate a video stream having the second special effect,
transmitting in real-time the video stream having the second special effect via the network, and
causing the live streaming interface to be displayed on the plurality of client devices, wherein the live streaming interface includes the video stream having the second special effect.

8. The system of claim 1 , wherein receiving the trigger includes:
receiving the trigger from a third-party application, wherein the trigger is generated by the third-party application based on an occurrence within the third-party application.

9. The system of claim 8 , wherein
the third-party application is an email application, a messaging application, or a telephone application, and
the occurrence within the third-party application is a new email notification, a new message notification, or a telephone call notification.

10. The system of claim 8 , wherein the third-party application is the video game.

11. The system of claim 10 , wherein the occurrence within the video game includes a predetermined action by or on the user's character.

12. A method of triggering changes to real-time special effects included in a live streaming video comprising:
transmitting in real-time to a plurality of client devices a video stream of a user captured by a camera, the video stream of the user comprising images of the user;
causing a live streaming interface to be displayed on the plurality of client devices, wherein the live streaming interface includes the video stream and a live stream of the video game being played by the user in real-time, wherein the video stream and the live stream of a video game are displayed independently in the live streaming interface;
receiving a trigger to apply one of a plurality of special effects to the video stream, wherein each of the plurality of special effects is an augmented reality special effect using a combination of a visual effect applied to the user's face in the video stream and a sound effect applied to audio of the video stream;
determining a first special effect of the plurality of special effects is associated with the trigger;
applying in real-time the first special effect to the video stream to generate a video stream having the first special effect;
transmitting in real-time the video stream having the first special effect via the network, and
causing the live streaming interface including the video stream having the first special effect to be displayed on the plurality of client devices.

13. The method of claim 12 , wherein receiving the trigger includes:
receiving a selection of a display item included in the live streaming interface from one of the plurality of client devices.

14. The method of claim 13 , wherein determining the first special effect is associated with the trigger includes:
determining that the first special effect is associated with the display item.

15. The method of claim 12 , further comprising:
causing a lens selection user interface to be displayed on a display device, the lens selection user interface including a plurality of lens selections,
wherein the plurality of lens selections are associated with a plurality of special effects, respectively,
wherein the plurality of lens selections include a first lens selection associated with the first special effect and a second lens selection associated with a second special effect,
wherein receiving the trigger includes:
receiving an input from a user via the lens selection user interface selecting the first lens selection, or receiving an input from the user of a set of keys on a keyboard, wherein the set of keys is associated with the first lens selection.

16. The method of claim 12 , further comprising:
receiving a subsequent trigger to apply one of a plurality of special effects to the video stream,
determining a second special effect of the plurality of special effects is associated with the subsequent trigger,
applying in real-time the second special effect to the video stream to generate a video stream having the second special effect,
transmitting in real-time the video stream having the second speci

Assignees

• Snap Inc

Inventors

• Gaiduchenko Artem
• Yerofieiev Artem
• Pozharskyi Bohdan
• Lupin Gabriel
• Kholovchuk Oleksii
• Chen Travis
• Monastyrshyn Yurii
• Makoviichuk Denys

Classifications

• H04L65/762 Primary

  at the source  (reformatting of additional data in video distribution servers H04N21/2355) · CPC title

• H04L65/61

  for supporting one-way streaming services, e.g. Internet radio · CPC title

• H04L65/403 Primary

  Arrangements for multi-party communication, e.g. for conferences (data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15) · CPC title

• H04L65/602 Primary

  Electricity · mapped topic

• H04L65/4069

  Electricity · mapped topic