Video segmentation and searching by segmentation dimensions

Patent metadata

Field	Value
Publication number	US-10560734-B2
Application number	US-201715492972-A
Country	US
Kind code	B2
Filing date	Apr 20, 2017
Priority date	Aug 1, 2016
Publication date	Feb 11, 2020
Grant date	Feb 11, 2020

Abstract

In various embodiments, methods and systems for implementing video segmentation are provided. A video management system implements a video segment manager that supports generating enhanced segmented video. Enhanced segmented video is a time-based segment of video content. Enhanced segmented video is generated based on a video content cognitive index, segmentation dimensions, segmentation rules and segment reconstruction rules. The video content cognitive index is built for indexing video content. Segmentation rules are applied to the video content to break the video content into time-based segments, the time-based segments are associated with corresponding segmentation dimensions for the video content. Segment reconstruction rules are then applied to the time-based segments to reconstruct the time-based segments into enhanced segmented video. The enhanced segmented video and corresponding values of the segmentation dimensions can be leveraged as distinct portions of the video content for different types of functionality in the video management system.

First claim

The invention claimed is:

1. A system for implementing video management, the system comprising:
one or more hardware processors and memory storing computer-executable instructions and components embodied thereon that, when executed, by the one or more hardware processors, cause the hardware processors to:
generate a video content cognitive index based on video content and a plurality of segmentation dimensions;
for a plurality of points within the video content, quantify the video content for each of the plurality of segmentation dimensions via one or more video analyzers such that each point within the video content has a value for each of the plurality of segmentation dimensions;
segment the video content into a plurality of time-based segments;
upon segmenting the video content into the plurality of time-based segments, generate an enhanced segmented video including one or both of a segment reconstruction rule to merge two or more time-based segments shorter than a first threshold length into a longer time-based segment, and a segment reconstruction rule to split a time-based segment longer than a second threshold length into two or more shorter time-based segments;
receive a search query for video content;
based on values of the plurality of segmentation dimensions for each of the plurality of time-based segments, identify one or more time-based segments relevant to the search query; and
generate a video segment playlist from the one or more time-based segments relevant to the search query.

2. The system of claim 1 , wherein generating the video content cognitive index is based on a video analyzer service that analyzes the video content to identify video elements associated with the plurality of segmentation dimensions.

3. The system of claim 1 , wherein generating the video content cognitive index is based on analyzing the video content into parallel time-based dimensions for the plurality of segmentation dimensions.

4. The system of claim 1 , wherein the segment reconstruction rules are based at least in part on the plurality of segmentation dimensions of the video content cognitive index.

5. A computer-implemented method for video management, the method comprising:
accessing video content using a video content cognitive index, wherein the video content is associated with a plurality of segmentation dimensions such that different portions of the video content have different values for each of the plurality of segmentation dimensions;
segmenting the video content into a plurality of time-based segments;
upon segmenting the video content into the plurality of time-based segments, generating an enhanced segmented video based on segment reconstruction rules including one or both of a segment reconstruction rule to merge two or more time-based segments shorter than a first threshold length into a longer time-based segment, and a segment reconstruction rule to split a time-based segment longer than a second threshold length into two or more shorter time-based segments;
receiving a search query for video content; and
based on values of the plurality of segmentation dimensions for each of the plurality of time-based segments, identifying one or more time-based segments relevant to the search query.

6. The method of claim 5 , wherein the video content cognitive index is generated based on a video analyzer service that analyzes the video content into parallel time-based dimensions to identify video elements associated with the plurality of segmentation dimensions.

7. The method of claim 5 , wherein the segment reconstruction rules are based at least in part on the plurality of segmentation dimensions of the video content cognitive index.

8. One or more computer storage media having computer-executable instructions embodied thereon that, when executed, by one or more processors, causes the one or more processors to perform a method for video management, the method comprising:
accessing video content;
generating a video content cognitive index based on video content and a plurality of segmentation dimensions, such that different portions of the video content have different values for each of the plurality of segmentation dimensions;
segmenting the video content into a plurality of time-based segments;
upon segmenting the video content into the plurality of time-based segments, generating an enhanced segmented video based on segment reconstruction rules including one or both of a segment reconstruction rule to merge two or more time-based segments shorter than a first threshold length into a longer time-based segment, and a segment reconstruction rule to split a time-based segment longer than a second threshold length into two or more shorter time-based segments;
receiving a search query for video content; and
based on values of the plurality of segmentation dimensions for each of the plurality of time-based segments, identifying one or more time-based segments relevant to the search query.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Jassin Ohad
• Levi Avner
• Nir Oron
• ZIV Ori

Classifications

• H04N21/26258 Primary

  for generating a list of items to be played back in a given order, e.g. playlist, or scheduling item distribution according to such list (retrieval of multimedia data based on playlists G06F16/40) · CPC title

• H04N21/4828

  for searching programme descriptors (retrieval of video data G06F16/739) · CPC title

• H04N21/23418

  involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59) · CPC title

• H04N21/234345

  the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment · CPC title

• H04N21/8456

  by decomposing the content in the time domain, e.g. in time segments · CPC title