Device specific multi-channel data compression

Patent metadata

Field	Value
Publication number	US-9875747-B1
Application number	US-201615211417-A
Country	US
Kind code	B1
Filing date	Jul 15, 2016
Priority date	Jul 15, 2016
Publication date	Jan 23, 2018
Grant date	Jan 23, 2018

Abstract

A sensor device may include a computing device in communication with multiple microphones. A neural network executing on the computing device may receive audio signals from each microphone. One microphone signal may serve as a reference signal. The neural network may extract differences in signal characteristics of the other microphone signals as compared to the reference signal. The neural network may combine these signal differences into a lossy compressed signal. The sensor device may transmit the lossy compressed signal and the lossless reference signal to a remote neural network executing in a cloud computing environment for decompression and sound recognition analysis.

First claim

The invention claimed is:

1. A method comprising:
determining, by at least a first neural network layer of a neural network of a first device, a first signal difference between a signal characteristic of a first audio signal and a signal characteristic of a second audio signal, wherein the first signal difference includes a difference in a frequency response;
compressing, by at least a second neural network layer of the neural network and based on the first signal difference, the first audio signal and the second audio signal into a third audio signal; and
providing, by the first device to a second device, the first audio signal and the third audio signal.

2. The method of claim 1 , further comprising:
determining, by at least the first neural network layer, a plurality of signal differences between one or more signal characteristics of the first audio signal and one or more signal characteristics of the second audio signal; and
selecting, by the neural network of the first device, the first signal difference from among the plurality of signal differences.

3. The method of claim 1 , further comprising:
receiving, by the first device from a first audio signal source, the first audio signal; and
receiving, by the first device from a second audio signal source, the second audio signal,
wherein the first audio signal source comprises a first microphone and the second audio signal source comprises a second microphone distinct from the first microphone.

4. The method of claim 1 , further comprising:
receiving, by the first device from a first audio signal source, the first audio signal; and
receiving, by the first device from a second audio signal source, the second audio signal,
wherein the first audio signal source comprises a first microphone, the second audio signal source comprises a second microphone distinct from the first microphone, and the first microphone and the second microphone are disposed at distinct locations on the first device.

5. The method of claim 1 , further comprising:
receiving, by the first device from a first audio signal source, the first audio signal; and
receiving, by the first device, a plurality of audio signals from a plurality of audio signal sources other than the first audio signal source.

6. The method of claim 1 , further comprising:
receiving, by the first device from a first audio signal source, the first audio signal;
receiving, by the first device, a plurality of audio signals from a plurality of audio signal sources other than the first audio signal source; and
determining, by at least the first neural network layer, a plurality of signal differences between one or more signal characteristics of the first audio signal and one or more signal characteristics of the plurality of audio signals.

7. The method of claim 1 , further comprising:
receiving, by the first device from a first audio signal source, the first audio signal;
receiving, by the first device, a plurality of audio signals from a plurality of audio signal sources other than the first audio signal source;
determining, by at least the first neural network layer, a plurality of signal differences between one or more signal characteristics of the first audio signal and one or more signal characteristics of the plurality of audio signals; and
generating, by at least the second neural network layer based on the plurality of signal differences, the third audio signal.

8. The method of claim 1 , wherein the first audio signal comprises a lossless signal and the third audio signal comprises an audio signal generated by lossy compression.

9. The method of claim 1 , further comprising:
losslessly compressing, by the first device, the first audio signal.

10. The method of claim 1 , wherein a bit rate of the first audio signal is greater than a bit rate of the third audio signal.

11. The method of claim 1 , wherein the first neural network layer and the second neural network layer are distinct neural network layers of the neural network of the first device.

12. The method of claim 1 , wherein the neural network of the first device comprises at least one selected from the group consisting of a deep neural network, convolutional neural network, long short-term memory neural network, and a convolutional, long short-term memory, fully connected deep neural network.

13. The method of claim 1 , wherein the first signal difference comprises at least one selected from the group consisting of: a difference in phase, a difference in magnitude, and a difference in gain.

14. The method of claim 1 , wherein the first signal difference comprises at least one selected from the group consisting of: a transfer function of the first audio signal source and a transfer function of the second audio signal source.

15. The method of claim 1 , wherein the first neural network layer comprises a plurality of nodes.

16. The method of claim 1 , wherein a total number of nodes of the first neural network layer is greater than a total number of nodes of the second neural network layer.

17. The method of claim 1 , wherein the second neural network layer comprises exactly one node.

18. The method of claim 1 , wherein the neural network defines one or more cell states.

19. The method of claim 1 , wherein the neural network comprises three or more layers and there is no layer between the second neural network layer and the output of the neural network.

20. The method of claim 1 , wherein the second device is distinct and remote from the first device.

21. A non-transitory, computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations comprising:
determining, by at least a first neural network layer of a neural network of a first device, a first signal difference between a signal characteristic of a first audio signal and a signal characteristic of a second audio signal, wherein the first signal difference includes a difference in a frequency response;
compressing, by at least a second neural network layer of the neural network and based on the first signal difference, the first audio signal and the second audio signal into a third audio signal; and
providing, by the first device to a second device, the first audio signal and the third audio signal.

22. A first device comprising:
a processor; and
a non-transitory, computer-readable medium in communication with the processor and storing instructions that, when executed by the processor, cause the processor to perform operations comprising:
determining, by at least a first neural network layer of a neural network of a first device, a first signal difference between a signal characteristic of a first audio signal and a signal characteristic of a second audio signal, wherein the first signal difference includes a difference in a frequency response;
compressing, by at least a second neural network layer of the neural network and based on the first signal difference, the first audio signal and the second audio signal into a third audio signal; and
providing, to a second device, the first audio signal and the third audio signal.

23. A method comprising:
generating, by a first device and based on a first audio signal and a second audio signal, a third audio signal;
determining, by at least a first neural network layer of a neural network of the first device, a first signal difference between a signal characteristic of the first audio signal and a signal charac

Assignees

• Google Inc
• Google Llc

Inventors

• Kim Chanwoo
• Nongpiur Rajeev Conrad
• Sainath Tara

Classifications

• G10L19/008 Primary

  Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing · CPC title

• G10L19/0017

  Lossless audio signal coding; Perfect reconstruction of coded audio signal by transmission of coding error (G10L19/24 takes precedence) · CPC title

• G10L25/30

  using neural networks · CPC title

• G10L25/72

  for transmitting results of analysis · CPC title