Video categorization method and apparatus, and storage medium

Patent metadata

Field	Value
Publication number	US-10115019-B2
Application number	US-201615241804-A
Country	US
Kind code	B2
Filing date	Aug 19, 2016
Priority date	Dec 1, 2015
Publication date	Oct 30, 2018
Grant date	Oct 30, 2018

Abstract

A video may be categorized into a picture category or a video category. A key frame of the video includes a face and a face feature in the key frame is obtained. Face features respectively associated with a plurality of picture categories are acquired and the video is assigned to one of the picture categories based on a comparison of the key frame face feature and the face features of the picture categories. Videos may first be associated with a video category by comparing key frame face features from the videos, and then the video category may be assigned to a picture category based on comparison of a video category face feature with a plurality of picture category face features. Alternatively, a video may be assigned to a picture category based on matching capture times and capture locations between the video and a reference picture in the picture category.

First claim

We claim:

1. A video categorization method, the method comprising:
acquiring a key frame from a video, the key frame comprising an image including a face;
acquiring a face feature from the key frame of the video;
acquiring one or more face features that correspond to one or more respective picture categories;
selecting a picture category to which the video belongs based on the face feature from the key frame and the one or more face features corresponding to the one or more respective picture categories;
assigning the video to the picture category to which the video belongs;
wherein the acquiring the key frame from the video comprises:
acquiring, from the video, at least one video frame comprising one or more faces,
determining a face parameter for each of the at least one video frame, the face parameter comprising either or both of a face quantity and a face location, and
determining the key frame from the video, based on the face parameter for each of the at least one video frame; and

wherein determining the key frame in the video according to the face parameter for each of the at least one video frame comprises:
determining, according to the face parameter for each of the at least one video frame, one or more non-duplicate video frames each having a face parameter that does not reoccur for other video frames, and
selecting at least one of the non-duplicate video frames as the key frame.

2. The method according to claim 1 , wherein the determining the key frame in the video according to the face parameter for each of the at least one video frame comprises:
determining, according to the face parameter for each of the at least one video frame, at least one group of duplicate video frames having a same face parameter, each group of duplicate video frames comprising at least two video frames, wherein a difference between capturing times of a latest captured video frame and an earliest captured video frame in each group of duplicate video frames is less than or equal to a predetermined time duration, and all the video frames in each group of duplicate video frames has a same face parameter; and
selecting any one of each group of duplicate video frames as the key frame.

3. The method according to claim 1 , wherein selecting the picture category to which the video belongs comprises:
in instances when there are at least two videos to categorize, determining a face feature in a key frame of each of the videos;
performing face clustering for the at least two videos based on the face feature in the key frame of each of the at least two videos, to identify at least one video category; and
selecting a picture category for each of the at least one video category, where each picture category corresponds to the same face feature as does a corresponding one of the at least one video category, each selection based on comparison of a face feature that corresponds to a respective one of the at least one video category and the one or more face features corresponding to the one or more picture categories, and
the assigning the video to the picture category to which the video belongs comprises:
assigning each video in each respective one of the at least one video category to the picture category that corresponds to a same face feature as corresponds to the respective one of the at least one video category.

4. The method according to claim 1 , wherein the selecting the picture category to which the video belongs comprises:
determining, from the one or more picture categories that correspond to the one or more face features, a picture category that corresponds to a face feature that matches the face feature in the key frame; and
identifying the matching picture category as the picture category to which the video belongs.

5. The method according to claim 1 , wherein the method further comprises:
acquiring a capturing time and a capturing location of the video;
determining a reference picture which has a same capturing time and capturing location as the video; and
assigning the video to a picture category to which the reference picture belongs.

6. A video categorization apparatus, the video categorization apparatus comprising:
a processor; and
a memory for storing instructions executable by the processor, wherein the processor is configured to:
acquire a key frame from a video, the key frame comprising an image including a face;
acquire a face feature from the key frame of the video;
acquire one or more face features that correspond to one or more respective picture categories;
select a picture category to which the video belongs based on the face feature from the key frame and the one or more face features corresponding to the one or more respective picture categories;
assign the video to the picture category to which the video belongs;
acquire, from the video, at least one video frame comprising one or more faces;
determine a face parameter for each of the at least one video frame, the face parameter comprising either or both of a face quantity and a face location;
determine the key frame from the video, based on the face parameter for each of the at least one video frame;
determine, according to the face parameter for each of the at least one video frame, one or more non-duplicate video frames each having a face parameter that does not reoccur for other video frames; and
select at least one of the non-duplicate video frames as the key frame.

7. The apparatus of claim 6 , wherein the processor is further configured to:
determine, according to the face parameter for each of the at least one video frame, at least one group of duplicate video frames having the same face parameter, each group of duplicate video frames comprising at least two video frames, wherein a difference between capturing times of a latest captured video frame and an earliest captured video frame in each group of duplicate video frames is less than or equal to a predetermined time duration, and all the video frames in each group of duplicate video frames have the same face parameter; and
select any one of each group of duplicate video frames as the key frame.

8. The apparatus of claim 6 , wherein the processor is further configured to:
in instances when there are at least two videos to categorize, determine a face feature in a key frame in each of the videos;
perform face clustering for the at least two videos based on the face feature in the key frame of each of the at least two videos, to identify at least one video category;
select a picture category for each of the at least one video category, where each picture category corresponds to a same face feature as does a corresponding one of the least one video category, each selection based on a comparison of a face feature that corresponds to a respective one of the at least one video category and the one or more face features corresponding to the one or more respective picture categories; and
assign each video in each respective one of the at least one video category to the picture category which corresponds to the same face feature as does the respective one of the at least one video category.

9. The apparatus of claim 6 , wherein the processor is further configured to:
determine, from the one or more picture categories that correspond to the one or more face features, a picture category that corresponds to a face feature that matches the face feature in the key frame; and
identify the matching picture category as the picture category to which the video belongs.

10. A non-transitory computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor of a mobile terminal, causes the mobile terminal to:
ac

Assignees

• Xiaomi Inc

Inventors

• Chen Zhijun
• Hou Wendi
• Long Fei

Classifications

• G06V20/41

  Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49) · CPC title

• G06F16/784 Primary

  the detected or recognised objects being people · CPC title

• G06F18/24

  Classification techniques · CPC title

• G06F18/23

  Clustering techniques · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title