Annotating a video with a personalized recap video based on relevancy and watch history

Patent metadata

Field	Value
Publication number	US-12260639-B2
Application number	US-202117454666-A
Country	US
Kind code	B2
Filing date	Nov 12, 2021
Priority date	Nov 12, 2021
Publication date	Mar 25, 2025
Grant date	Mar 25, 2025

Abstract

A method for automatically annotating an intended video with at least one personalized recap video based on previously viewed videos is provided. The method may include automatically tracking user viewership of the previously viewed videos, and in response to detecting an intention to view the intended video: automatically identifying and extracting a subset of video footage from one or more of the previously viewed videos based on the tracked user viewership and based on a determined relevancy of the subset of video footage to content in the intended video; generating the at least one personalized recap video by compiling and sorting the extracted subset of video footage from the one or more previously viewed videos into a compilation video; and annotating the intended video with the at least one personalized video by presenting the at least one personalized recap video on the intended video.

First claim

What is claimed is:

1. A computer-implemented method for automatically annotating an intended video with at least one personalized recap video based on previously viewed videos, comprising:
automatically tracking user viewership of the previously viewed videos, and in response to detecting an intention to view the intended video, automatically generating the at least one personalized recap video based on a configurable amount of time elapsed since the user viewership of a previously viewed video immediately preceding the intended video, wherein automatically generating the at least one personalized recap video further comprises:
automatically identifying and extracting a subset of video footage from one or more of the previously viewed videos based on the tracked user viewership and based on a determined relevancy of the subset of video footage to unwatched content in the intended video, wherein automatically identifying and extracting the subset of video footage further comprises automatically identifying a topic and an entity appearing at various points in the unwatched content, and extracting the subset of video footage matching and comprising the identified topic and the entity from one or more previously viewed videos;
generating the at least one personalized recap video by compiling and sorting the extracted subset of video footage from the one or more previously viewed videos into a compilation video; and
annotating the intended video with the at least one personalized video by presenting the at least one personalized recap video on the intended video.

2. The computer-implemented method of claim 1 , wherein the previously viewed videos and the intended video are part of an episodic series of videos, wherein the previously viewed videos precede the intended video, and wherein the intended video comprises an unviewed video.

3. The computer-implemented method of claim 2 , further comprising:
in response to receiving the episodic series of videos:
transcribing audio from each video associated with the episodic series of videos using a speech-to-text algorithm to individually produce an audio transcript of each video with timestamp data; and
for each video, identifying entities and objects appearing at various points in a video using an image recognition algorithm and correlating the identified entities and objects with the transcribed audio and timestamp data for the video, wherein the identified entities identify characters in the episodic series of videos.

4. The method of claim 3 , further comprising:
for each video associated with the episodic series of videos, using machine learning to perform topic modeling on each video based on the audio transcript and the identified entities and objects to identify topics and changes in topics for each video, wherein the topics represents scenes in each video, and wherein the changes in topics represent changes in scenes for each video.

5. The computer-implemented method of claim 4 , wherein automatically identifying and extracting the subset of video footage from one or more of the previously viewed videos based on the tracked user viewership and based on the determined relevancy of the subset of video footage to the unwatched content in the intended video further comprises:
determining a match between the topics and the identified entities between the one or more previously viewed videos and the intended video, wherein the determined match is based on a threshold confidence score.

6. The computer-implemented method of claim 1 , wherein annotating the intended video with the at least one personalized video further comprises:
presenting the at least one personalized recap video as an introduction on the intended video.

7. The computer-implemented method of claim 1 , further comprising:
generating a plurality of personalized recap videos for the intended video; and
intersplicing each personalized recap video associated with the plurality of personalized recap videos at different times on the intended video based on the determined relevancy of each personalized recap video to a scene in the intended video.

8. A computer system for automatically annotating an intended video with at least one personalized recap video based on previously viewed videos, comprising:
one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage devices, and program instructions stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, wherein the computer system is capable of performing a method comprising:
automatically tracking user viewership of the previously viewed videos, and in response to detecting an intention to view the intended video, automatically generating the at least one personalized recap video based on a configurable amount of time elapsed since the user viewership of a previously viewed video immediately preceding the intended video, wherein automatically generating the at least one personalized recap video further comprises:
automatically identifying and extracting a subset of video footage from one or more of the previously viewed videos based on the tracked user viewership and based on a determined relevancy of the subset of video footage to unwatched content in the intended video, wherein automatically identifying and extracting the subset of video footage further comprises automatically identifying a topic and an entity appearing at various points in the unwatched content, and extracting the subset of video footage matching and comprising the identified topic and the entity from one or more previously viewed videos;
generating the at least one personalized recap video by compiling and sorting the extracted subset of video footage from the one or more previously viewed videos into a compilation video; and
annotating the intended video with the at least one personalized video by presenting the at least one personalized recap video on the intended video.

9. The computer system of claim 8 , wherein the previously viewed videos and the intended video are part of an episodic series of videos, wherein the previously viewed videos precede the intended video, and wherein the intended video comprises an unviewed video.

10. The computer system of claim 9 , further comprising:
in response to receiving the episodic series of videos:
transcribing audio from each video associated with the episodic series of videos using a speech-to-text algorithm to individually produce an audio transcript of each video with timestamp data; and
for each video, identifying entities and objects appearing at various points in a video using an image recognition algorithm and correlating the identified entities and objects with the transcribed audio and timestamp data for the video, wherein the identified entities identify characters in the episodic series of videos.

11. The computer system of claim 10 , further comprising:
for each video associated with the episodic series of videos, using machine learning to perform topic modeling on each video based on the audio transcript and the identified entities and objects to identify topics and changes in topics for each video, wherein the topics represents scenes in each video, and wherein the changes in topics represent changes in scenes for each video.

12. The computer system of claim 11 , wherein automatically identifying and extracting the subset of video footage from one or more of the previously viewed videos based on the tracked user viewership and based on the determined relevancy of the subset of video footage to the unwatched content in the

Assignees

• IBM

Inventors

• Frost Keith Gregory
• Tkac Vincent
• Myers Andrew C
• Rice Joshua M

Classifications

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06V20/48

  Matching video sequences · CPC title

• G06V20/46

  Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames · CPC title

• G10L15/26

  Speech to text systems (G10L15/08 takes precedence) · CPC title

• H04N21/44204

  Monitoring of content usage, e.g. the number of times a movie has been viewed, copied or the amount which has been watched (monitoring of user activities for profile generation for accessing a video database G06F16/739; protecting generic digital content where the protection is independent of the precise nature of the content G06F21/10; arrangements for monitoring the use made of the broadcast services in broadcast systems H04H60/31) · CPC title