Methods and systems for dynamic summary queue generation and provision

Patent metadata

Field	Value
Publication number	US-11729479-B2
Application number	US-202117165410-A
Country	US
Kind code	B2
Filing date	Feb 2, 2021
Priority date	Dec 31, 2020
Publication date	Aug 15, 2023
Grant date	Aug 15, 2023

Abstract

A stream of a media item is transmitted from a server computing system to one or more client computing systems. The media item has a timeline for playback of the media item. Participant-generated content is received at the server computing system from one or more participants at any of the one or more client computing systems. The participant-generated content is time-indexed to the timeline of the media item. The participant-generated content is processed to identify key portions of the media item based on indications present within the participant-generated content. A dynamic summary queue is generated that includes the key portions of the media item. The dynamic summary queue is assembled for streaming. A request to view the dynamic summary queue is received from a requestor. In response to the request, a stream of the dynamic summary queue is transmitted to a computing device of the requestor.

First claim

What is claimed is:

1. A method for generating a summary of a media item, comprising:
transmitting a stream of a media item from a server computing system to one or more client computing systems, the media item having a timeline for playback of the media item;
receiving participant-generated content at the server computing system from one or more participants at any of the one or more client computing systems, the participant-generated content time-indexed to the timeline of the media item;
processing the participant-generated content to identify key portions of the media item based on indications present within the participant-generated content;
generating a dynamic summary queue that includes the key portions of the media item, the dynamic summary queue assembled for streaming;
receiving a request to view the dynamic summary queue from a requestor;
transmitting a stream of the dynamic summary queue to a computing device of the requestor in response to the request;
receiving additional participant-generated content at the server computing system from one or more participants at any of the one or more client computing systems as the stream of the media item is transmitted, the additional participant-generated content time-indexed to the timeline of the media item; and
in response to receiving the additional participant-generated content, automatically changing some content already present within the dynamic summary queue based on the additional participant-generated content while continuing to transmit the stream of the dynamic summary queue.

2. The method as recited in claim 1 , wherein the media item is an audio-visual presentation.

3. The method as recited in claim 2 , wherein the audio-visual presentation is of one or more of a movie, a television show, a real-world sporting event, an electronic sporting event, an online show, an online video, and an uploaded video.

4. The method as recited in claim 1 , wherein the media item is a stream of game video and audio of a video game.

5. The method as recited in claim 4 , wherein the server computing system is a cloud gaming system configured to execute the video game in accordance with inputs received from respective controller devices of the one or more participants at the one or more client computing systems, the cloud gaming system configured to direct transmission of the stream of game video and audio reflecting play of the video game by the one or more participants, wherein the one or more participants provide the participant-generated content during play of the video game.

6. The method as recited in claim 4 , wherein the video game is an electronic sports video game.

7. The method as recited in claim 1 , wherein the participant-generated content includes one or more of text data, voice data, audio data, emoticon data, image data, camera-captured video data, camera-captured image data, tag data, and controller device input data.

8. The method as recited in claim 1 , wherein processing the participant-generated content includes extracting features from the participant-generated content to obtain a set of extracted features, classifying the extracted features into one or more classifications to obtain a set of classified features, and processing the set of classified features through a machine learning model to identify the key portions of the media item.

9. The method as recited in claim 8 , further comprising:
processing state data of the media item through the machine learning model in conjunction with processing the set of classified features through the machine learning model to identify the key portions of the media item, the state data of the media item correlated by time-index to the set of classified features.

10. The method as recited in claim 9 , wherein the state data of the media item includes state data of a video game executed in accordance with inputs received from respective controller devices of the one or more participants to generate the media item as a stream of game video and audio.

11. The method as recited in claim 1 , wherein the dynamic summary queue includes an auto-generated verbal summary and/or an auto-generated written summary of the key portions of the media item.

12. The method as recited in claim 1 , wherein the dynamic summary queue includes at least some of the participant-generated content associated with the key portions of the media item.

13. The method as recited in claim 1 , wherein the requestor is any of the one or more participants.

14. The method as recited in claim 1 , wherein the requestor is a person invited to join the one or more participants in watching the media item.

15. The method as recited in claim 1 , wherein the requestor is a person invited to join the one or more participants in playing a video game executed in accordance with inputs received from respective controller devices of the one or more participants to generate the media item as a stream of game video and audio.

16. The method as recited in claim 1 , further comprising:
receiving control parameters for generating the dynamic summary queue.

17. The method as recited in claim 16 , wherein the control parameters include specification of one or more types of indications present within the participant-generated content that are indicative of key portions of the media item.

18. The method as recited in claim 17 , wherein the types of indications include one or more a response, an expression, a gesture, an action, a statement, a sound, and/or a communication made by any of the one or more participants in relation to subject matter within the media item.

19. The method as recited in claim 16 , wherein the control parameters include specification of one or more types of events that occur within the media item.

20. The method as recited in claim 16 , wherein the control parameters include specification of which of the one or more participants that the participant-generated content is to be limited to when processing the participant-generated content to identify key portions of the media item.

21. The method as recited in claim 16 , wherein the control parameters include specification of a time window within the timeline for playback of the media item over which the dynamic summary queue is to be generated.

22. The method as recited in claim 16 , wherein the control parameters include specification of a recap resolution for the dynamic summary queue.

23. The method as recited in claim 1 , further comprising:
automatically adding new content to the dynamic summary queue based on the additional participant-generated content as the stream of the dynamic summary queue is transmitted.

24. The method as recited in claim 1 , further comprising:
automatically removing some content from the dynamic summary queue based on the additional participant-generated content as the stream of the dynamic summary queue is transmitted.

25. A system for generating a summary of a video, comprising:
program instructions stored in a computer memory for transmitting a stream of a media item from a server computing system to one or more client computing systems, the media item having a timeline for playback of the media item;
program instructions stored in the computer memory for receiving participant-generated content at the server computing system from one or more participants at any of the one or more client computing systems, the participant-generated content time-indexed to

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Benedetto Warren M

Classifications

• H04N21/8549 Primary

  Creating video summaries, e.g. movie trailer {(retrieval in video databases by using presentations in form of a video summary G06F16/739)} · CPC title

• A63F13/355

  Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client · CPC title

• G11B27/19

  by using information detectable on the record carrier · CPC title

• H04N21/235

  Processing of additional data, e.g. scrambling of additional data or processing content descriptors · CPC title

• H04N21/2393

  involving handling client requests (scheduling or organising the servicing of application requests H04L67/60) · CPC title