System and method for providing a video with lyrics overlay for use in a social messaging environment

Patent metadata

Field	Value
Publication number	US-10354633-B2
Application number	US-201715858507-A
Country	US
Kind code	B2
Filing date	Dec 29, 2017
Priority date	Dec 30, 2016
Publication date	Jul 16, 2019
Grant date	Jul 16, 2019

Abstract

In accordance with an embodiment, described herein is a system and method for providing a live lyrics overlay in a social messaging environment. The system can utilize advances in three-dimensional mapping technology that allow social messaging services, to offer real time video lenses or overlays to their users, and extends this three-dimensional mapping technology to support for lyrics. During creation of a video with lyrics lens overlay, the lyrics corresponding to a selected song are retrieved from a lyrics source, and are displayed within the video. For example, with the lyrics lens, a user can record an image of themselves on live video, singing along to a song clip, with the lyrics of the song displayed as if they appear to be coming from their mouths. The created live lyrics content can also be shared with other users of a social messaging environment.

First claim

What is claimed is:

1. A system for providing a live lyrics overlay in a social messaging environment, comprising:
one or more computers, including a media server executing thereon which receives requests from client media devices for media content;
wherein each client media device includes
a user interface for displaying media content options associated with items of media content at the media server, and receiving selections of media content options,
a media application that enables communication by the client media device with the media server, and
a messaging application that enables access to a social messaging environment;

wherein an item of media content provided by the media server is associated with a link or reference by which the item of media content can be retrieved from the media server, for playback at a client media device;
wherein during creation and recording of a video content at the client media device,
a selection of a media content associated with an audio clip and a song metadata is received at the client media device, wherein the song metadata is associated with a selected song retrieved from the media server,
in response to selection of the media content associated with the audio clip and the song metadata, lyrics associated with the selected song are identified and retrieved from a lyrics source, and are displayed at the user interface of the client media device within the video content as a lyrics lens overlay; and

wherein in response to the selection of the media content associated with the audio clip and the song metadata during creation and recording of the video content at the client media device, a shareable file or other data comprising the recorded video content and the lyric lens overlay is created, for playback by one or more recipient media devices.

2. The system of claim 1 , wherein the recorded video content with lyrics lens overlay is provided to one or more recipient media devices associated with user recipients, and the system is configured to use the song metadata associated therewith to play the recorded video content with lyrics lens overlay at the one or more recipient media devices, and enable access to the media content identified by the song metadata, at the media server.

3. The system of claim 1 , wherein the messaging application enables playing of the video content with displayed lyrics in a social messaging environment.

4. The system of claim 1 , wherein the client media device is configured to receive, from the media server, a media content recommendation to be used as the selected song in creating the video content.

5. The system of claim 1 , wherein the lyrics lens overlay is mapped to the video content using three-dimensional mapping technology.

6. The system of claim 1 , wherein the shareable file or other data is associated with a location or other information provided by sensors at the client media device.

7. A method for providing a live lyrics overlay in a social messaging environment, comprising:
using a media server executing on one or more computers to stream or otherwise provide access to media content items, which media server receives requests from client media devices for the media content items;
wherein each client media device includes
a user interface for displaying media content options associated with items of media content at the media server, and receiving selections of media content options,
a media application that enables communication by the client media device with the media server, and
a messaging application that enables access to a social messaging environment;

wherein an item of media content provided by the media server is associated with a link or reference by which the item of media content can be retrieved from the media server, for playback at a client media device;
during creation and recording of a video content at the client media device,
a selection of a media content associated with an audio clip and a song metadata is received at the client media device, wherein the song metadata is associated with a selected song retrieved from the media server,
in response to selection of the media content associated with the audio clip and the song metadata, lyrics associated with the selected song are identified and retrieved from a lyrics source, for display at the user interface of the client media device within the video content as a lyrics lens overlay; and

wherein in response to the selection of the media content associated with the audio clip and the song metadata during creation and recording of the video content at the client media device, a shareable file or other data comprising the recorded video content and the lyric lens overlay is created, for playback by one or more recipient media devices.

8. The method of claim 7 , wherein the recorded video content with lyrics lens overlay is provided to one or more recipient media devices associated with user recipients, and the system is configured to use the song metadata associated therewith to play the recorded video content with lyrics lens overlay at the one or more recipient media devices, and enable access to the media content identified by the song metadata, at the media server.

9. The method of claim 7 , wherein the messaging application enables playing of the video content with displayed lyrics in a social messaging environment.

10. The method of claim 7 , wherein the client media device is configured to receive, from the media server, a media content recommendation to be used as the selected song in creating the video content.

11. The method of claim 7 , wherein the lyrics lens overlay is mapped to the video content using three-dimensional mapping technology.

12. The method of claim 7 , wherein the shareable file or other data is associated with a location or other information provided by sensors at the client media device.

13. A non-transitory computer readable storage medium, including instructions stored thereon which when read and executed by one or more computers cause the one or more computers to perform the steps comprising:
using a media server executing on one or more computers to stream or otherwise provide access to media content items, which media server receives requests from client media devices for the media content items;
wherein each client media device includes
a user interface for displaying media content options associated with items of media content at the media server, and receiving selections of media content options,
a media application that enables communication by the client media device with the media server, and
a messaging application that enables access to a social messaging environment;

wherein an item of media content provided by the media server is associated with a link or reference by which the item of media content can be retrieved from the media server, for playback at a client media device;
during creation and recording of a video content at the client media device,
a selection of a media content associated with an audio clip and a song metadata is received at the client media device, wherein the song metadata is associated with a selected song retrieved from the media server,
in response to selection of the media content associated with the audio clip and the song metadata, lyrics associated with the selected song are identified and retrieved from a lyrics source, for display at the user interface of the client media device within the video content as a lyrics lens overlay; and

wherein in response to the selection of the media content associated with the audio clip and the song metadata during creation and recording of the video content at

Assignees

• Spotify Ab

Inventors

• O'Driscoll Brendan
• Sliney Aidan
• Watson Craig

Classifications

• G06Q10/40

  Business processes related to social networking or social networking services · CPC title

• G10H2220/011

  Lyrics displays, e.g. for karaoke applications · CPC title

• G06F16/9535

  Search customisation based on user profiles and personalisation · CPC title

• G06F3/0482

  Interaction with lists of selectable items, e.g. menus · CPC title

• H04L51/10

  Multimedia information · CPC title