Method for distinguishing components of signal of environment

Patent metadata

Field	Value
Publication number	US-9685155-B2
Application number	US-201615147382-A
Country	US
Kind code	B2
Filing date	May 5, 2016
Priority date	Jul 7, 2015
Publication date	Jun 20, 2017
Grant date	Jun 20, 2017

Abstract

A method distinguishes components of a signal by processing the signal to estimate a set of analysis features, wherein each analysis feature defines an element of the signal and has feature values that represent parts of the signal, processing the signal to estimate input features of the signal, and processing the input features using a deep neural network to assign an associative descriptor to each element of the signal, wherein a degree of similarity between the associative descriptors of different elements is related to a degree to which the parts of the signal represented by the elements belong to a single component of the signal. The similarities between associative descriptors are processed to estimate correspondences between the elements of the signal and the components in the signal. Then, the signal is processed using the correspondences to distinguish component parts of the signal.

First claim

We claim:

1. A method for distinguishing one or more components of a signal, comprising steps:
acquiring the signal from an environment with a sensor;
processing the signal to estimate a set of analysis features, wherein each analysis feature defines an element of the signal, and has feature values that represent parts of the signal;
processing the signal to estimate input features of the signal;
processing the input features using a deep neural network (DNN) to assign an associative descriptor to each element of the signal, wherein a degree of similarity between the associative descriptors of different elements is related to a degree to which the parts of the signal represented by the elements belong to a single component of the signal;
processing the similarities between associative descriptors to estimate correspondences between the elements of the signal and one or more components in the signal; and
processing the signal using the correspondences to distinguish the parts of the one or more components of the signal, wherein the steps are performed in a processor.

2. The method of claim 1 , wherein the signal is processed to change an intensity of the elements corresponding to the one or more components of the signal.

3. The method of claim 1 , wherein the DNN is a recurrent neural network.

4. The method of claim 1 , wherein the neural network is a convolutional neural network.

5. The method of claim 1 , wherein an association between the associative descriptor and the one or more components is estimated using K-means clustering.

6. The method of claim 1 , wherein an association between the associative descriptor and the one or more components is estimated using Gaussian mixture models.

7. The method of claim 1 , wherein an association between the associative descriptor and the one or more components is estimated using a singular value decomposition.

8. The method of claim 1 , wherein an association between the associative descriptors is processed to form a graph, and graph-based signal processing is performed on the signal.

9. The method of claim 1 , wherein an analysis feature transformation that defines the elements is optimized using training data to reduce an error in distinguishing the one or more components of the signal.

10. The method of claim 1 , wherein the DNN is optimized using training data to produce the associative descriptors such that the processing of the similarities between associative descriptors to estimate correspondences between the elements of the signal and one or more components in the signal reduces an error in distinguishing the components of the signal.

11. The method of claim 1 , wherein one or more of the components of the signal are speech, and the processing of the signal produces a separate signal corresponding to one or more speech signals.

12. The method of claim 1 , wherein the one or more components of the signal are used for recognizing objects.

13. The method of claim 12 , wherein one or more of the separate signal is processed in a speech recognition system to recognize words spoken by a speaker.

14. The method of claim 1 , wherein the one or more components are distinguished using a different measure of similarity to estimate the correspondences.

15. The method of claim 14 , wherein the one or more components are organized in a hierarchy and correspondences at one or more levels of the hierarchy are estimated using the different measures of similarity.

16. The method of claim 1 , wherein one or more classes are represented using class descriptors, wherein a measure of similarity between an element descriptor and a class descriptor is related to a degree to which the element belongs to the class.

17. The method of claim 1 , wherein a video signal that is associated with the signal is processed to assign the associative descriptors to the elements in the video signal, and correspondences between the one or more components and the elements of the signal and the video signal are estimated jointly.

18. The method of claim 1 , wherein text that is associated with the signal is processed to assign the associative descriptors to the elements in the text, and correspondences between the one or more components and the elements of the signal and the text are estimated jointly.

19. A non-transitory computer readable memory embodied thereon a program executable by a processor for performing a method for distinguishing one or more components of a signal, the method comprising:
acquiring the signal from an environment using a sensor;
processing the signal to estimate a set of analysis features, wherein each analysis feature defines an element of the signal, and has feature values that represent parts of the signal;
processing the signal to estimate input features of the signal;
processing the input features using a deep neural network (DNN) to assign an associative descriptor to each element of the signal, wherein a degree of similarity between the associative descriptors of different elements is related to a degree to which the parts of the signal represented by the elements belong to a single component of the signal;
processing the similarities between associative descriptors to estimate correspondences between the elements of the signal and one or more components in the signal; and
processing the signal using the correspondences to distinguish the parts of the one or more components of the signal.

Assignees

• Mitsubishi Electric Res Laboratories Inc

Inventors

• Hershey John
• Le Roux Jonathan
• Watanabe Shinji
• Chen Zhuo

Classifications

• G06V10/77

  Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation · CPC title

• G06V10/26

  Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion · CPC title

• G06V10/82 Primary

  using neural networks · CPC title

• G06V10/763

  Non-hierarchical techniques, e.g. based on statistics of modelling distributions · CPC title

• G10L15/16 Primary

  using artificial neural networks · CPC title