Content based video content segmentation

Patent metadata

Field	Value
Publication number	US-9888279-B2
Application number	US-201414483507-A
Country	US
Kind code	B2
Filing date	Sep 11, 2014
Priority date	Sep 13, 2013
Publication date	Feb 6, 2018
Grant date	Feb 6, 2018

Abstract

A method receives video content and metadata associated with video content. The method then extracts features of the video content based on the metadata. Portions of the visual, audio, and textual features are fused into composite features that include multiple features from the visual, audio, and textual features. A set of video segments of the video content is identified based on the composite features of the video content. Also, the segments may be identified based on a user query.

First claim

What is claimed is:

1. A method comprising:
in a video data analyzer of a first computing device, configuring an extraction, based on metadata associated with video content, of content features;
wherein the content features are selected from the group consisting of visual features of the video content, audio features of the video content, and textual features of the video content,
wherein one or more feature extractors corresponding to the content features are selected from the group consisting of a visual feature extractor for content features selected from visual features of the video content, an audio feature extractor for content features selected from audio features of the video content, and a text feature extractor for content features selected from textual features of the video content, and
wherein configuring the extraction comprises configuring the one or more selected feature extractors to extract the respective content features in accordance with one or more operating parameters that are used internally by the respective feature extractor, and that are tunable by the video data analyzer to alter an extraction behavior of the feature extractor based on the metadata;

creating a single data stream of fused information for rendering in a client computing device communicatively coupled to one or more distributed content servers, wherein the creating comprises:
fusing, in a plurality of fusion modules communicatively coupled to the one or more distributed content servers, portions of the content features into composite features that are generated from functions of the multiple features from the content features;
identifying, by one or more of the plurality of fusion modules, a plurality of video segments comprising one or more video segments of the video content based on the composite features; and
rendering the created single data stream, in a user interface of the client computing device, by rendering representations of the identified video segments.

2. The method of claim 1 , wherein some of the plurality of video segments are identified based on only one content feature.

3. The method of claim 1 , wherein identifying the plurality of video segments comprises combining non-contiguous segments from the video content into a segment.

4. The method of claim 1 , wherein the multiple features are based on at least two of the group consisting of visual features of the video content, audio features of the video content, and textual features of the video content.

5. The method of claim 1 , wherein:
the composite features include the multiple features from at least two of the visual feature extractor, the audio feature extractor, and the text feature extractor.

6. The method of claim 1 , wherein:
the extraction is performed by a plurality of extractors, and
the metadata is used to configure an extractor in the plurality of extractors to extract one of visual, audio, and textual features based on the metadata.

7. The method of claim 1 , wherein:
the identifying is performed by a plurality of fusion modules, and
the metadata is used to configure a fusion module in the plurality of fusion modules to fuse the multiple features into the composite features.

8. The method of claim 7 , wherein the fusion module determines a composite feature based on the metadata.

9. The method of claim 1 , further comprising classifying the plurality of video segments based on the metadata.

10. The method of claim 1 , wherein the metadata comprises program metadata received from an electronic program guide data source.

11. The method of claim 1 , further comprising:
displaying the plurality of video segments;
receiving a selection of one of the plurality of video segments; and
displaying the one of the plurality of video segments.

12. The method of claim 11 , further comprising adding supplemental content in association with the one of the plurality of video segments based on a feature associated with the one of the plurality of video segments.

13. The method of claim 12 , wherein the supplemental content is based on a type of user reaction to the one of the plurality of video segments.

14. An apparatus comprising:
a plurality of computer processors comprising a video data analyzer processor and one or more segment services processors;
at least one non-transitory computer readable storage memory coupled to each of the plurality of computer processors and comprising instructions that when executed by one or more of the computer processors cause the one or more of the computer processors to be configured for:
in the video data analyzer processor, configuring an extraction, based on metadata associated with video content, of content features;
wherein the content features are selected from the group consisting of visual features of the video content, audio features of the video content, and textual features of the video content,
wherein one or more feature extractors corresponding to the content features are selected from the group consisting of a visual feature extractor for content features selected from visual features of the video content, an audio feature extractor for content features selected from audio features of the video content, and a text feature extractor for content features selected from textual features of the video content, and
wherein configuring the extraction comprises configuring the one or more selected feature extractors to extract the respective content features in accordance with one or more operating parameters that are used internally by the respective feature extractor, and that are tunable by the video data analyzer to alter an extraction behavior of the feature extractor based on the metadata;

creating a single data stream of fused information for rendering in a client computing device communicatively coupled to one or more distributed content servers, wherein the creating comprises:
in a plurality of fusion modules in the segment services processors, fusing portions of the content features into composite features that include are generated from functions of the multiple features from the content features, wherein the segment services processors are communicatively coupled to the one or more distributed content servers;
identifying, by one or more of the plurality of fusion modules, a plurality of video segments comprising one or more video segments of the video content based on the composite features; and
rendering the created single data stream, in a user interface of the client computing device, by rendering representations of the identified video segments.

15. A method for creating a single data stream of fused information for rendering in a client computing device communicatively coupled to one or more distributed content servers, the method comprising:
receiving a search query comprising at least one word;
receiving a textual program index associated with each video program from a plurality of video programs stored on a content server;
identifying, by one or more of a plurality of fusion modules, matching video programs from the plurality of video programs based on the textual program index associated with each video program and the at least one word;
receiving a user selection of a matching video program from the matching video programs to identify a selected video program;
receiving a plurality of text records associated with the selected video program;
searching, by one or more of the plurality of fusion modules, the text records of the selected video program to identify matching text records based on the at least one word;
segmenting, by one or more of the plurali

Assignees

• Arris Entpr Inc
• Arris Entpr Llc

Inventors

• Ishtiaq Faisal
• Fonseca Jr Benedito J
• Baum Kevin L
• Braskich Anthony J
• Emeott Stephen P
• Gandhi Bhavan
• Li Renxiang
• Smith Alfonso Martinez
• Needham Michael L
• Ould Dellahy Isselmou

Classifications

• H04N21/2353

  specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata · CPC title

• H04N21/4316 Primary

  for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window · CPC title

• G06K9/00711

  Physics · mapped topic

• G06K2209/27

  Physics · mapped topic

• H04N21/4884

  for displaying subtitles · CPC title