Model-based media classification service using sensed media noise characteristics

Patent metadata

Field	Value
Publication number	US-2017193097-A1
Application number	US-201615185654-A
Country	US
Kind code	A1
Filing date	Jun 17, 2016
Priority date	Jan 3, 2016
Publication date	Jul 6, 2017
Grant date	—

Abstract

A neural network-based classifier system can receive a query including a media signal and, in response, provide an indication that the query corresponds to a specified media type or media class. The neural network-based classifier system can select and apply various models to facilitate media classification. In an example embodiment, a query can be analyzed for various characteristics, such as a noise profile, before it is input to the network-based classifier. If the query has greater than a specified threshold noise characteristic, then a successful classification can be unlikely and a classification process based on the query can be terminated before computational resources are expended. Query signals that meet or exceed a threshold condition can be provided to the network-based classifier for media classification. In an example embodiment, a remote device or a central media classifier circuit can determine a noise profile for a query.

First claim

What is claimed is:

1 . A method for classifying media, the method comprising:
accessing, using one or more processor circuits associated with a first device, digital media data that represents a media query to be identified; determining, using the one or more processor circuits associated with the first device, a noise characteristic corresponding to the digital media data; and if the determined noise characteristic indicates less than a specified threshold amount of noise corresponding to the digital media data, then transmitting the digital media data to a remote media classification circuit that is configured to identify a source characteristic of the digital media data, and otherwise inhibiting the transmitting the digital media data to the remote media classification circuit.

2 . The method of claim 1 , wherein the accessing the digital media data includes receiving an audio signal using a microphone of a mobile device, and wherein the determining the noise characteristic includes determining a noise characteristic of the received audio signal using the one or more processor circuits.

3 . The method of claim 1 , further comprising:
accessing, using the one or more processor circuits associated with the first device, a first context parameter that corresponds to the media query to be identified; determining, using the one or more processor circuits associated with the first device, a signal quality characteristic corresponding to the first context parameter; and if the determined signal quality characteristic corresponding to the first context parameter is less than a specified threshold signal quality, then inhibiting the transmitting the digital media data to the remote media classification circuit.

4 . The method of claim 1 , further comprising retrieving the specified threshold amount of noise from a database of threshold noise characteristics, the database established by prior training of the same or similar media queries with a neural network-based classifier system, the database stored at the first device or at the remote media classification circuit.

5 . The method of claim 1 , wherein the transmitting the digital media data to the media classification circuit includes providing the digital media data to an input of a convolutional neural network classifier system.

6 . The method of claim 1 , wherein if the determined noise characteristic corresponds to a first specified noise threshold range, then initiating a first media classification process having a first search depth using the media classification circuit, and if the determined noise characteristic corresponds to a second greater noise threshold range, then initiating a second media classification process having a different second search depth using the media classification circuit.

7 . The method of claim 1 , further comprising selecting a media classification search depth based on the determined noise characteristic corresponding to the digital media data, wherein the selected media classification search depth indicates a maximum processing time elapsed or a maximum processing effort expended by the media classification circuit to identify the source characteristic of the digital media data.

8 . The method of claim 1 , further comprising:
comparing, using the remote media classification circuit, the determined noise characteristic corresponding to the digital media data with noise characteristics corresponding to successfully classified other media data and unsuccessfully classified other media data; and if the determined noise characteristic more closely corresponds to the noise characteristics corresponding to successfully classified other media data than to the noise characteristics corresponding to unsuccessfully classified other media data, then transmitting the digital media data to the media classification circuit.

9 . The method of claim 1 , wherein the inhibiting the transmitting the digital media data to the media classification circuit includes accessing subsequent digital media data that represents a different media query to be identified, determining an updated noise characteristic corresponding to the subsequent digital media data, and determining whether the updated noise characteristic indicates less than the specified threshold amount of noise.

10 . The method of claim 1 , wherein the accessing the digital media data includes accessing an audio sample, using the first device, corresponding to the media query;
wherein the determining the noise characteristic includes determining a noise characteristic for the audio sample; and wherein the transmitting the digital media data to the media classification circuit includes transmitting all or a portion of the audio sample to the media classification circuit.

11 . The method of claim 1 , wherein the accessing the digital media data includes accessing a video signal sample corresponding to the media query;
wherein the determining the noise characteristic includes determining a visual characteristic of the video signal sample; and wherein the transmitting the digital media data to the media classification circuit includes transmitting all or a portion of the video signal sample to the media classification circuit.

12 . The method of claim 1 , further comprising receiving, at the first device and from the media classification circuit, an indication of a source characteristic of the digital media data, and displaying the indication of the source characteristic of the digital media data to a user of the mobile device.

13 . A non-transitory computer-readable storage medium comprising instructions that, when executed by at least one processor of a machine, cause the machine to perform operations comprising:
accessing, using one or more processor circuits, digital media data that represents a media query to be identified; determining, using the one or more processor circuits, a likelihood that the media query can be successfully identified by a neural network classifier based on a spectral characteristic of the digital media data; and if the determined likelihood is greater than a specified threshold likelihood, then:
providing the digital media data to a first input of the neural network classifier;
receiving from the neural network classifier, in response to the digital media data, a media type probability index for the media query; and
providing information about the media type probability index to a remote device to provide an indication of a media type to a user of the remote device.

14 . The non-transitory computer-readable storage medium of claim 13 , wherein the operations further comprise determining a signal noise characteristic corresponding to the digital media data, and wherein the determining the likelihood that the media query can be successfully identified by the neural network classifier includes determining the likelihood based on the determined signal noise characteristic.

15 . The non-transitory computer-readable storage medium of claim 13 , wherein the operations further comprise determining a frequency content characteristic corresponding to the digital media data, and wherein the determining the likelihood that media query can be successfully identified by the neural network classifier includes determining the likelihood based on the determined frequency content characteristic.

16 . The non-transitory computer-readable storage medium of claim 13 , wherein the operations further comprise accessing a context parameter associated with the media query, wherein the context parameter and the digital media data are

Assignees

• Gracenote Inc

Inventors

• Cramer Jason
• Cremer Markus K
• Popp Phillip
• Summers Cameron Aubrey

Classifications

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06F16/35 Primary

  Clustering; Classification · CPC title

• G06F16/61

  Indexing; Data structures therefor; Storage structures · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title