Method for obtaining a mega-frame image fingerprint for image fingerprint based content identification, method for identifying a video sequence, and corresponding device

Patent metadata

Field	Value
Publication number	US-2016110609-A1
Application number	US-201414786983-A
Country	US
Kind code	A1
Filing date	Apr 25, 2014
Priority date	Apr 25, 2013
Publication date	Apr 21, 2016
Grant date	—

Abstract

A temporal section that is defined by boundary images is selected in a video sequence. A maximum of k stable image frames are selected in the temporal section of image frames having a lowest temporal activity. Image fingerprints are computed from the selected stable image frames. A mega-frame image fingerprint data structure is constructed from the computed fingerprints.

First claim

1 - 12 . (canceled)

13 . A method for obtaining a mega-frame image fingerprint from a temporal section of a video sequence for fingerprint based identification of a video sequence, comprising:
selecting a temporal section defined by boundary image frames in the video sequence, said boundary image frames delimiting a sequence of image frames in the video sequence; selecting a maximum of k stable image frames j in the selected temporal section, by computing a sum of similarity distances between a number of neighbor image frames of a candidate stable image frame j in the selected temporal section and determining the k minimum computed sums of similarity distances in the temporal section, while respecting an interspacing of at least n image frames between the stable image frames j; for each of the selected maximum k stable image frames j, selecting an image frame within a selection window of a width of M frames, the selection window being centered in the selected stable image frame j, the selected image frame replacing the selected stable image frame j; and for each of the selected maximum k stable image frames j, computing an image fingerprint, and constructing a mega-frame image fingerprint data structure that comprises the computed image fingerprints.

14 . The method according to claim 13 , wherein said boundary image frames are detected by analyzing a distance between digest vectors computed over successive image frames of said video sequence, a boundary image frame being detected when said distance between said digest vectors exceeds a threshold.

15 . The method according to claim 13 , wherein said image frame selected in said step of selecting an image frame within a selection window is an I-frame.

16 . The method according to claim 13 , wherein said image frame selected in said step of selecting an image frame within a selection window is an image frame of which a luminous exposure is within defined limits.

17 . The method according to claim 13 , further comprising enhancing said data structure with metadata comprising information related to a temporal position of the fingerprints in the data structure with regard to the video sequence.

18 . The method according to claim 13 , wherein said data structure is stored as an aggregated set of image fingerprints.

19 . A method for identifying a video sequence, wherein it comprises:
selecting a temporal section of the video sequence defined by boundary image frames in the video sequence, said boundary image frames delimiting a sequence of image frames in the video sequence; selecting a maximum of k stable image frames in the selected temporal section, by computing of a sum of similarity distances between a number of neighbor image frames of a candidate stable image frame j in the selected temporal section and determining the k minimum computed sums of similarity distances in the temporal section, while respecting an interspacing of at least n image frames between the stable image frames; for each of the selected maximum k stable image frames j, computing an image fingerprint, and constructing a mega-frame image fingerprint data structure that comprises the computed image fingerprints; for each of the selected maximum k stable image frames j, selecting an image frame within a selection window of a width of M frames, the selection window being centered in the selected stable image frame j, the selected image frame replacing the selected stable image frame j;
comparing the constructed mega-frame image fingerprint data structure with mega-frame image fingerprint data structures from an image fingerprint data base; and
said video sequence being identified by one of said data structures in said data base, if upon said comparing a data structure is found in said data base that corresponds to said constructed data structure.

20 . The method according to claim 19 , wherein said comparing is done according to a Nearest Neighbor Search method.

21 . The method according to claim 19 , wherein said comparing is done according to a Locality Sensitive Hashing search method.

22 . The method according to claim 19 , wherein said comparing is done according to a Product Quantization search method.

23 . A device for obtaining a mega-frame image fingerprint from a temporal section of a video sequence, comprising:
a temporal section selector configured to select a temporal section of the video sequence, the temporal section being defined by boundary image frames in the video sequence, the boundary image frames delimiting a sequence of image frames; a stable frame selector configured to select a maximum of k stable image frames j in the selected temporal section, by computing of a sum of similarity distances between a number of neighbor image frames of a candidate stable image frame j in the selected temporal section and determining the k minimum computed sums of similarity distances in the temporal section, while respecting a interspacing of at least n image frames between the stable image frames j; a best frame selector configured to select, for each of the selected maximum k stable image frames j, an image frame within a selection window of a width of M frames, the selection window being centered in the selected stable image frame j, the selected image frame replacing the selected stable image frame j; a data structure constructor configured to compute an image fingerprint for each of the selected maximum k stable image frames j, and configured to construct a mega-frame image fingerprint data structure that comprises the computed image fingerprints.

24 . A device for identifying a video sequence, the device comprising:
a temporal section selector configured to select a temporal section of the video sequence defined by boundary image frames in the video sequence, said boundary image frames delimiting a sequence of image frames in the video sequence; a stable frame selector configured to select a maximum of k stable image frames in the selected temporal section, by computing a sum of similarity distances between a number of neighbor image frames of a candidate stable image frame j in the selected temporal section and determining the k minimum computed sums of similarity distances in the temporal section, while respecting an interspacing of at least n image frames between the stable image frames; a best frame selector configured to select, for each of the maximum k determined stable image frames, an image frame within a selection window of a width of M frames, the selection window being centered in the selected stable image frame j, the selected image frame replacing the selected stable image frame j; a data structure constructor configured to compute an image fingerprint for each of the determined maximum k stable image frames j, and for constructing of a mega-frame image fingerprint data structure that comprises the computed image fingerprints; a data structure comparator configured to compare the constructed mega-frame image fingerprint data structure with mega-frame image fingerprint data structures from an image fingerprint data base; and said video sequence being identified by one of said data structures in said data base, if upon said comparing a data structure is found in said data base that corresponds to said constructed data structure.

25 . The method according to claim 13 , wherein said image frame selected in said step of selecting an image frame within a selection window is an I-frame with a luminous exposure that is within defined limits.

26 . The method according to claim 19 , wherein said image frame selected in said step of selecting an image fram

Assignees

• Thomson Licensing

Inventors

• Lefebvre Frederic
• Zepeda Salvatierra Joaquin
• Perez Patrick

Classifications

• G06K9/6202

  Physics · mapped topic

• G06K9/00744 Primary

  Physics · mapped topic

• G06V20/46 Primary

  Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames · CPC title