Commercial detection based on audio fingerprinting

Patent metadata

Field	Value
Publication number	US-9258604-B1
Application number	US-201414552039-A
Country	US
Kind code	B1
Filing date	Nov 24, 2014
Priority date	Nov 24, 2014
Publication date	Feb 9, 2016
Grant date	Feb 9, 2016

Abstract

A commercial detection system generates a probe audio fingerprint of an audio signal associated with a media stream on a broadcast channel and determines whether the media stream has commercial content based on analysis of the probe audio fingerprint. The commercial detection system determines whether a same match between the probe audio fingerprint and a reference audio fingerprint is observed across multiple broadcast channels. Responsive to the number of same matches exceeding a predetermined threshold, the commercial detection system determines that the media stream has commercial content. The commercial detection system may also apply a trained feature analysis model to extracted acoustic features of the audio signal. The commercial detection system determines whether the media stream has commercial content based on a confidence score assigned to the probe audio fingerprint. The commercial detection system reduces false positive detection using program guide information of the media stream.

First claim

What is claimed is:

1. A computer-implemented method comprising:
receiving an audio signal including a plurality of frames, each frame representing a portion of the audio signal, the audio signal associated with a media stream on a broadcast channel;
generating a probe audio fingerprint based on one or more of the plurality of frames;
comparing the probe audio fingerprint with one or more reference audio fingerprints from a plurality of reference audio fingerprints, a reference audio fingerprint of the plurality of reference audio fingerprints representing a set of acoustic features of an audio signal associated with commercial content;
determining whether the probe audio fingerprint matches a reference audio fingerprint of the plurality of reference audio fingerprints based on comparison of the probe audio fingerprint and the reference audio fingerprint; and
responsive to a match between the probe audio fingerprint and the reference audio fingerprint of the plurality of reference audio fingerprints:
determining whether an audio fingerprint associated with a media stream broadcast over a channel of a plurality of other broadcast channels matches the reference audio fingerprint: and
determining that the media stream associated with the probe audio fingerprint has commercial content in response to the number of matches between the audio fingerprints associated with the plurality of other broadcast channels and the reference audio fingerprint exceeding a predetermined threshold.

2. The computer-implemented method of claim 1 , further comprising:
responsive to a match between the probe audio fingerprint and the reference audio fingerprint of the plurality of reference audio fingerprints:
determining whether an audio fingerprint associated with a media stream of a plurality of other media streams matches the reference audio fingerprint;
determining that the media stream associated with the probe audio fingerprint has commercial content in response to the number of matches between the audio fingerprints associated with the plurality of other media streams and the reference audio fingerprint exceeding a predetermined threshold.

3. The computer-implemented method of claim 1 , further comprising:
receiving program guide information associated with the media stream on the broadcast channel, the program guide information describing scheduling information of known commercials broadcast along with the media stream on the broadcast channel; and
verifying the commercial content determination based on the program guide information.

4. The computer-implemented method of claim 1 , further comprising:
receiving program guide information associated with other media streams on the plurality of other broadcast channels, the program guide information describing scheduling information of known commercials broadcast along with the other media streams on the other broadcast channels.

5. The computer-implemented method of claim 1 , further comprising:
extracting a plurality of acoustic features from the audio signal associated with the media stream;
applying a trained analysis model to the extracted acoustic features of the audio signal;
assigning a confidence score to the probe audio signal based on the analysis of the extracted acoustic features of the media stream;
detecting commercial content in the media stream based on the confidence score assigned to the probe audio signal.

6. The computer-implemented method of claim 5 , wherein the plurality of acoustic features of the audio signal comprise a plurality of acoustic features of the audio signal in frequency domain comprising:
magnitude spectrum of the audio signal;
Mel-frequency cepstral coefficients of the audios signal;
spectral bandwidth of the audio signal;
spectral flatness measure of the audio signal;
spectral fluctuation of the audio signal;
number of extreme value frequencies; and
number of silent frequencies.

7. The computer-implemented method of claim 5 , wherein the plurality of acoustic features of the audio signal comprise a plurality of acoustic features of the audio signal in time domain comprising mean, standard deviation and covariance matrix of features vectors of the audio signal, the feature vectors of the audio signal being observed over a predefined time window.

8. The computer-implemented method of claim 5 , wherein the confidence score assigned to the probe audio fingerprint indicates a likelihood that the media stream associated with the probe audio fingerprint has commercial content.

9. The computer-implemented method of claim 5 , wherein assigning a confidence score to the probe audio signal comprises:
determining a degree of similarities between the acoustic features associated with the probe audio fingerprint and the acoustic features of reference audio fingerprints of known commercial content.

10. A non-transitory computer-readable storage medium storing computer program instructions, executed by a computer processor, for detecting commercial in a media stream, the computer program instructions comprising instructions for:
receiving an audio signal including a plurality of frames, each frame representing a portion of the audio signal, the audio signal associated with a media stream on a broadcast channel;
generating a probe audio fingerprint based on one or more of the plurality of frames;
comparing the probe audio fingerprint with one or more reference audio fingerprints from a plurality of reference audio fingerprints, a reference audio fingerprint of the plurality of reference audio fingerprints representing a set of acoustic features of an audio signal associated with commercial content;
determining whether the probe audio fingerprint matches a reference audio fingerprint of the plurality of reference audio fingerprints based on comparison of the probe audio fingerprint and the reference audio fingerprint; and
responsive to a match between the probe audio fingerprint and the reference audio fingerprint of the plurality of reference audio fingerprints:
determining whether an audio fingerprint associated with a media stream broadcast over a channel of a plurality of other broadcast channels matches the reference audio fingerprint; and
determining that the media stream associated with the probe audio fingerprint has commercial content in response to the number of matches between the audio fingerprints associated with the plurality of other broadcast channels and the reference audio fingerprint exceeding a predetermined threshold.

11. The computer-readable storage medium of claim 10 , further comprising computer program instructions for:
responsive to a match between the probe audio fingerprint and the reference audio fingerprint of the plurality of reference audio fingerprints:
determining whether an audio fingerprint associated with a media stream of a plurality of other media streams matches the reference audio fingerprint;
determining that the media stream associated with the probe audio fingerprint has commercial content in response to the number of matches between the audio fingerprints associated with the plurality of other media streams and the reference audio fingerprint exceeding a predetermined threshold.

12. The computer-readable storage medium of claim 10 , further comprising computer program instructions for:
receiving program guide information associated with the media stream on the broadcast channel, the program guide information describing scheduling information of known commercials broadcast along with the media stream on the broadcast channel; and
verifying the commercial content determination based on the program guide information.

Assignees

• Facebook Inc

Inventors

• Bilobrov Sergiy
• Poutivski Iouri

Classifications

• H04N21/44008

  involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59) · CPC title

• H04N21/4821

  using a grid, e.g. sorted out by channel and broadcast time · CPC title

• H04N21/8358

  involving watermark {(protecting executable software by watermarking G06F21/16; image watermarking in general G06T1/0021; watermarks inserted in still images for transmission purposes H04N1/32144; inserting watermarks during video coding H04N19/467)} · CPC title

• H04N21/466

  Learning process for intelligent management, e.g. learning user preferences for recommending movies {(services using the results of monitoring in broadcast systems H04H60/61)} · CPC title

• H04N21/482

  End-user interface for programme selection · CPC title