Predicting engagement in video content

Patent metadata

Field	Value
Publication number	US-9681158-B1
Application number	US-201514595975-A
Country	US
Kind code	B1
Filing date	Jan 13, 2015
Priority date	May 27, 2009
Publication date	Jun 13, 2017
Grant date	Jun 13, 2017

Abstract

User engagement in unwatched videos is predicted by collecting and aggregating data describing user engagement with watched videos. The data are normalized to reduce the influence of factors other than the content of the videos on user engagement. Engagement metrics are calculated for segments of watched videos that indicate user engagement with each segment relative to overall user engagement with the watched videos. Features of the watched videos within time windows are characterized, and a function is learned that relates the features of the videos within the time windows to the engagement metrics for the time windows. The features of a time window of an unwatched video are characterized, and the learned function is applied to the features to predict user engagement to the time window of the unwatched video. The unwatched video can be enhanced based on the predicted user engagement.

First claim

The invention claimed is:

1. A method of using a computer having one or more processors to predict user engagement, comprising:
determining, by the one or more processors, engagement metrics indicating user engagement with portions of a plurality of different watched videos;
characterizing, by the one or more processors, features from video and/or audio content within the portions of the plurality of different watched videos, the characterized features derived separately from the determined engagement metrics, wherein characterizing the features comprises:
identifying a time window of a watched video; and
computing a feature vector describing characteristics of content within the watched video within the identified time window;

determining, by the one or more processors, a relationship between the characterized features from the content within the portions of the plurality of different watched videos and the engagement metrics of the portions of the plurality of different watched videos;
characterizing, by the one or more processors, features from video and/or audio content within a portion of an unwatched video, the unwatched video not among the plurality of different watched videos;
predicting, by the one or more processors, an engagement metric for the portion of the unwatched video based at least in part on the characterized features from the content within the portion of the unwatched video and the determined relationship between the characterized features from the content within the portions of the plurality of different watched videos and the engagement metrics of the portions of the plurality of different watched videos; and
storing, by the one or more processors, the predicted engagement metric for the portion of the unwatched video.

2. The method of claim 1 , wherein determining engagement metrics indicating user engagement with portions of a plurality of different watched videos comprises:
receiving data describing user interactions with the plurality of different watched videos;
aggregating the data describing the user interactions with the plurality of different watched videos to determine user engagement with the portions of the plurality of different watched videos;
normalizing the aggregated data to increase an influence of the content within the portions of the plurality of different watched videos on the user engagement and to reduce an influence of factors other than the content within the portions of the plurality of different watched videos on the user engagement; and
determining the engagement metrics for the portions of the plurality of different watched videos responsive to the normalized aggregated data describing the user interactions with the plurality of different watched videos.

3. The method of claim 2 , wherein determining engagement metrics for the portions of the plurality of different watched videos comprises:
determining a mean user engagement across the plurality of different watched videos; and
calculating the engagement metrics for the portions of the plurality of different watched videos responsive to differences between the user engagements for the portions and the mean user engagement.

4. The method of claim 1 , wherein predicting the engagement metric comprises:
identifying a time window of the unwatched video;
computing a feature vector describing characteristics of content within the unwatched video within the identified time window; and
applying a machine-learned prediction function to the feature vector to predict user engagement with the content within the identified time window.

5. The method of claim 1 , wherein the feature vector describes one or more characteristics of content selected from the set consisting of:
frequency coefficients of a visual component of the content;
spatial arrangement of color of a visual component of the content;
visual texture of a visual component of the content; and
motion of pixels in a visual component of the content.

6. The method of claim 1 , wherein determining a relationship between the characterized features from the content within the portions of the plurality of different watched videos and the engagement metrics of the portions of the plurality of different watched videos comprises:
using machine learning to learn a function relating the characterized features of the content within the portions of the plurality of different watched videos to the engagement metrics of the portions of the plurality of different watched videos.

7. The method of claim 6 , wherein using machine learning comprises:
identifying portions of the plurality of different watched videos having very high and/or very low user engagement metrics relative to engagement metrics of other portions of the watched videos; and
training a classifier using the content within the identified portions of the plurality of different watched videos having very high and/or very low user engagement metrics by using the content within the identified portions as ground truth labels for the classifier;
wherein predicting an engagement metric for the portion of the unwatched video uses the classifier to make the prediction.

8. The method of claim 1 , further comprising:
enhancing the portion of the unwatched video responsive at least in part to the predicted engagement metric.

9. The method of claim 8 , wherein enhancing the portion of the unwatched video comprises:
overlaying an advertisement on the portion of the unwatched video.

10. The method of claim 8 , wherein enhancing the portion of the unwatched video comprises:
capturing and storing a representative image or video segment for the unwatched video from the portion of the unwatched video.

11. A non-transitory computer-readable storage medium storing executable computer program instructions for predicting user engagement, the computer program instructions comprising instructions for:
determining engagement metrics indicating user engagement with portions of a plurality of different watched videos;
characterizing features from video and/or audio content within the portions of the plurality of different watched videos, the characterized features derived separately from the determined engagement metrics;
determining a relationship between the characterized features from the content within the portions of the plurality of different watched videos and the engagement metrics of the portions of the plurality of different watched videos;
characterizing features from video and/or audio content within a portion of an unwatched video, the unwatched video not among the plurality of different watched videos;
predicting an engagement metric for the portion of the unwatched video based at least in part on the characterized features from the content within the portion of the unwatched video and the determined relationship between the characterized features from the content within the portions of the plurality of different watched videos and the engagement metrics of the portions of the plurality of different watched videos; and
storing the predicted engagement metric for the portion of the unwatched video.

12. The computer-readable storage medium of claim 11 , wherein determining engagement metrics indicating user engagement with portions of a plurality of different watched videos comprises:
receiving data describing user interactions with the plurality of different watched videos;
aggregating the data describing the user interactions with the plurality of different watched videos to determine user engagement with the portions of the plurality of different watched videos;
normalizing the aggregated data to increase an influence of the content within

Assignees

• Google Inc

Inventors

• Gargi Ullas
• Yagnik Jay N
• Sarkar Anindya

Classifications

• H04N21/44222

  Analytics of user selections, e.g. selection of programmes or purchase activity (monitoring of user selections in data processing systems G06F11/34; arrangements for monitoring the user's behaviour or opinions in broadcast systems H04H60/33) · CPC title

• H04N21/466

  Learning process for intelligent management, e.g. learning user preferences for recommending movies {(services using the results of monitoring in broadcast systems H04H60/61)} · CPC title

• H04N21/251

  Learning process for intelligent management, e.g. learning user preferences for recommending movies (details of learning user preferences for the retrieval of video data in a video database G06F16/739; computer systems using learning methods G06N3/08) · CPC title

• H04N21/23418 Primary

  involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59) · CPC title

• H04N21/25891

  being end-user preferences (retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18) · CPC title