Systems and methods implementing a machine learning architecture for video processing

Patent metadata

Field	Value
Publication number	US-12283084-B2
Application number	US-202418948428-A
Country	US
Kind code	B2
Filing date	Nov 14, 2024
Priority date	Jul 26, 2017
Publication date	Apr 22, 2025
Grant date	Apr 22, 2025

Abstract

The present disclosure describes a method comprising receiving a video; segmenting the video into a plurality of segments, each of the plurality of segments comprising a plurality of images; executing one or more machine learning models using the plurality of segments to generate a segment score for each of the plurality of segments, the segment score for a segment indicating a likelihood that a user will interact with the segment; generating a video performance score for the video as a function of the segment scores for the plurality of segments; and generating a record comprising the video performance score for the video and an identification of the video.

First claim

What is claimed is:

1. A method, comprising:
receiving, by one or more processors, a video;
segmenting, by the one or more processors, the video into a plurality of segments, each of the plurality of segments comprising a plurality of images;
executing, by the one or more processors, one or more machine learning models using the plurality of segments to generate a segment score for each of the plurality of segments, the segment score for a segment indicating a likelihood that a user will interact with the segment;
generating, by the one or more processors, a video performance score for the video as a function of the segment scores for the plurality of segments; and
generating, by the one or more processors, a record comprising the video performance score for the video and an identification of the video.

2. The method of claim 1 , wherein segmenting the video into a plurality of segments comprises segmenting, by the one or more processors, the video into the plurality of segments each having a defined length and having a defined overlap between pairs of sequential segments of the plurality of segments.

3. The method of claim 1 , further comprising:
identifying, by the one or more processors, edit points in the videos; and
segmenting, by the one or more processors, the plurality of segments based on the identified edit points.

4. The method of claim 1 , wherein executing the one or more machine learning model to generate the segment score for each of the plurality of segments comprises:
iteratively executing, by the one or more processors, a feature extraction machine learning model using the plurality of segments to generate a segment embedding for each of the plurality of segments; and
iteratively executing, by the one or more processors, a content scoring machine learning model based on the plurality of segments to generate the segment score for each of the plurality of segments.

5. The method of claim 1 , wherein generating the video performance score for the video comprises:
aggregating, by the one or more processors, the segment scores of the plurality of segments to generate the video performance score.

6. The method of claim 5 , wherein aggregating the segments scores comprises:
assigning, by the one or more processors, weights to the segment scores according to lengths of the segments corresponding to the segment scores; and
aggregating, by the one or more processors, the segment scores according to the assigned weights.

7. The method of claim 5 , wherein generating the video performance score for the video comprises:
assigning, by the one or more processors, weights to the segment scores according to distances of the segments corresponding to the segment scores from a beginning of the video; and
aggregating, by the one or more processors, the segment scores according to the assigned weights.

8. The method of claim 1 , further comprising:
ranking, by the one or more processors, the plurality of segments according to the segment performance scores of the plurality of segments; and
presenting, by the one or more processors, images from the plurality of segments on a user interface in order according to the rankings of the plurality of segments from which the images respectively originated.

9. The method of claim 1 , further comprising:
identifying, by the one or more processors, a defined number of segments with the lowest segment performance scores of the plurality of segments; and
removing, by the one or more processors, the defined number of segments with the lowest segment performance scores of the plurality of segments from the video.

10. The method of claim 1 , further comprising:
identifying, by the one or more processors, a highest scoring segment of the plurality of segments based on the segments scores for the plurality of segments;
extracting, by the one or more processors, one or more images from the highest scoring segment;
executing, by the one or more processors, at least one machine learning model to generate an image performance score for each of the one or more images extracted from the highest scoring segment;
identifying, by the one or more processors, a highest scoring image of the one or more images based on the generated image performance scores; and
generating, by the one or more processors, a record identifying the highest scoring image.

11. The method of claim 1 , further comprising:
identifying, by the one or more processors, a defined number of segments with the highest segment performance scores of the plurality of segments;
concatenating, by the one or more processors, the defined number of segments into a concatenated video; and
storing, by the one or more processors, the concatenated video in memory.

12. A system, comprising one or more processors coupled with memory and configured to:
receive a video;
segment the video into a plurality of segments, each of the plurality of segments comprising a plurality of images;
execute one or more machine learning models using the plurality of segments to generate a segment score for each of the plurality of segments;
generate a video performance score for the video as a function of the segment scores for the plurality of segments; and
generate a record comprising the video performance score for the video and an identification of the video.

13. The system of claim 12 , wherein the one or more processors are configured to segment the video into a plurality of segments by segmenting the video into the plurality of segments each having a defined length and having a defined overlap between pairs of sequential segments of the plurality of segments.

14. The system of claim 12 , wherein the one or more processors are further configured to:
determine segment content for each of the plurality of segments; and
segment the plurality of segments based on a change in segment content between pairs of sequential segments of the plurality of segments.

15. The system of claim 12 , wherein the one or more processors are further configured to execute the one or more machine learning model to generate the segment score for each of the plurality of segments by:
iteratively executing a feature extraction machine learning model using the plurality of segments to generate a segment embedding for each of the plurality of segments; and
iteratively executing a content scoring machine learning model using the plurality of segments to generate the segment score for each of the plurality of segments.

16. The system of claim 12 , wherein the one or more processors are configured to generate the video performance score for the video by:
aggregating the segment scores of the plurality of segments to generate the video performance score.

17. The system of claim 16 , wherein the one or more processors are configured to aggregate the segments scores by:
assigning weights to the segment scores according to lengths of the segments corresponding to the segment scores; and
aggregating the segment scores according to the assigned weights.

18. The system of claim 16 , wherein the one or more processors are configured to generate the video performance score for the video by:
assigning weights to the segment scores according to distances of the segments corresponding to the segment scores from a beginning of the video; and
aggregating the segment scores according to the assigned weights.

19. Non-transitory computer-readable media comprising instructions that, when executed by one or more processors, cause the one or more processors to:
rec

Assignees

• Vizit Labs Inc

Inventors

• Saraee Elham
• Hamedi Jehan
• Halloran Zachary

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06V10/40

  Extraction of image or video features · CPC title

• G06V10/82

  using neural networks · CPC title

• G06F16/438

  Presentation of query results · CPC title

• G06V20/46

  Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames · CPC title