Method, apparatus, and system for speech enhancement and separation based on audio and radio signals

Patent metadata

Field	Value
Publication number	US-12352882-B2
Application number	US-202217827902-A
Country	US
Kind code	B2
Filing date	May 30, 2022
Priority date	Jul 17, 2015
Publication date	Jul 8, 2025
Grant date	Jul 8, 2025

Abstract

Methods, apparatus and systems for radio-assisted signal estimation are described. In one example, a described system comprises: a sensor configured to obtain a baseband mixture signal in a venue; a transmitter configured to transmit a first radio signal through a wireless channel of the venue; a receiver configured to receive a second radio signal through the wireless channel; and a processor. The baseband mixture signal comprises a mixture of a first source signal and an additional signal. The first source signal is generated by a first motion of a first object in the venue. The second radio signal differs from the first radio signal due to the wireless channel and at least the first motion of the first object in the venue. The processor is configured for: obtaining a radio feature of the second radio signal, constructing a first adaptive filter for the baseband mixture signal based on the radio feature, filtering the baseband mixture signal using the first adaptive filter to obtain a first output signal, and generating an estimation of the first source signal based on the first output signal.

First claim

We claim:

1. A system for radio-assisted signal estimation, comprising:
a sensor configured to sense a baseband mixture signal in a venue, wherein:
the baseband mixture signal comprises a mixture of a first source signal and an additional signal,
the first source signal is generated by a first motion of a first object in the venue,
the baseband mixture signal is a baseband signal that has not been modulated to a radio frequency;

a transmitter configured to transmit a first radio signal through a wireless channel of the venue;
a receiver configured to receive a second radio signal through the wireless channel, wherein the second radio signal differs from the first radio signal due to the wireless channel and at least the first motion of the first object in the venue, wherein both the first radio signal and the second radio signal are not baseband signals; and
a processor configured for:
obtaining a baseband radio-derived signal based on a radio feature of the second radio signal,
configuring a first adaptive filter for the baseband mixture signal based on the baseband radio-derived signal, wherein configuring the first adaptive filter comprises:
configuring a first baseband filter for the baseband mixture signal, wherein the first baseband filter comprises: a first pre-processing in a first signal domain, a first transformation from the first signal domain to a first transformed domain, and a first transformed-domain filter in the first transformed domain,
configuring a second baseband filter for the baseband radio-derived signal, wherein the second baseband filter comprises: a second pre-processing in a second signal domain, a second transformation from the second signal domain to a second transformed domain, and a second transformed-domain filter in the second transformed domain, and
configuring the first adaptive filter based on the first baseband filter, the second baseband filter, a third filter and a fourth filter, wherein:
the third filter comprises: a third pre-processing based on outputs of the first and second baseband filters, and a third transformed-domain filter in a third signal domain,
the third pre-processing comprises combining a first vector output from the first baseband filter in the first transformed domain with a second vector output from the second baseband filter in the second transformed domain to generate a combined signal in the third signal domain, and
the fourth filter comprises: a fourth transformed-domain filter in the first transformed domain based on output of the third transformed-domain filter, a first inverse transformation from the first transformed domain to the first signal domain, and a post-processing in the first signal domain,

filtering the baseband mixture signal using the first adaptive filter to obtain a first output signal, and
generating an estimation of the first source signal based on the first output signal.

2. The system of claim 1 , wherein configuring the first adaptive filter comprises:
configuring the first baseband filter without using the radio feature of the second radio signal; and
configuring the second baseband filter based on the radio feature of the second radio signal, wherein the second baseband filter comprises at least one signal processing element similar to that of the first baseband filter.

3. The system of claim 1 , wherein:
at least one of the first pre-processing, the second pre-processing or the third pre-processing comprises at least one of: feature extraction, magnitude computation, high-pass filter, random rotation in IQ plane, phase cleaning, or magnitude cleaning;
at least one of the first signal domain or the second signal domain is a time domain; and
at least one of the first transformed domain or the second transformed domain is a time-frequency domain.

4. The system of claim 3 , wherein:
at least one of the first transformation or the second transformation comprises at least one of: an adaptive encoder, a short-time Fourier transform, a projection into the time-frequency domain, a number of kernels, a number of convolutional filters, or the number of convolutional filters followed by another number of convolutional filters;
at least one of the first transformed-domain filter, the second transformed-domain filter, the third transformed-domain filter, or the fourth transformed-domain filter comprises at least one of: reshaping, concatenation, neural network, dual-path recurrent neural network, neural network with bidirectional long short-term memory, or deep neural network;
the first inverse transformation comprises at least one of: an inverse of the first transformation, or a transposed convolution operation;
the post-processing comprises at least one of: concatenation, overlap-and-add (OLA), synchronized overlap-and-add (SOLA), overlap-and-save, overlap-and-discard, or overlap-and-scrap.

5. The system of claim 4 , wherein filtering the baseband mixture signal using the first adaptive filter comprises:
filtering the baseband mixture signal in the first signal domain using the first baseband filter to generate a first intermediate signal in the first transformed domain;
filtering the baseband radio-derived signal in the second signal domain using the second baseband filter to generate a second intermediate signal in the second transformed domain, wherein the baseband radio-derived signal is derived from the second radio signal based on the radio feature;
filtering a combination of the first intermediate signal and the second intermediate signal using the third filter to configure the fourth transformed-domain filter of the fourth filter; and
filtering a transformed baseband mixture signal using the fourth filter to obtain the first output signal, wherein the transformed baseband mixture signal is obtained by preprocessing the baseband mixture signal using the first pre-processing followed by transforming from the first signal domain to the first transformed domain using the first transformation.

6. The system of claim 5 , wherein:
the additional signal comprises a mixture of a second source signal and a third signal;
the second source signal is generated by a second motion of a second object in the venue;
the second radio signal differs from the first radio signal due to: the wireless channel, the first motion of the first object, and the second motion of the second object;
the processor is further configured for:
configuring a second adaptive filter for the baseband mixture signal based on an additional baseband radio-derived signal derived based on a second tab of each CI of a second beamformed TSCI of the radio feature, the second beamformed TSCI of the radio feature being associated with a second direction of the second object, the second tab being associated with a second range of the second object,
filtering the baseband mixture signal using the second adaptive filter to obtain a second output signal, and
generating an estimation of the second source signal based on the second output signal.

7. The system of claim 6 , wherein:
the processor is further configured for configuring the second adaptive filter based on the first baseband filter, the second baseband filter, an additional third filter and an additional fourth filter;
the additional third filter comprises: an additional third pre-processing based on outputs of the first and second baseband filters, and an additional third transformed-domain filter in the third signal domain; and
the additional fourth filter comprises: an additional fourth transformed-domain filter in the first transformed domain based on output of the additional third transformed-domain filter, the first inverse transformation from the first transformed domain to the first signal domain, and the post-processing i

Assignees

• Ozturk Muhammed Zahid
• Wu Chenshu
• Wang Beibei
• Liu K J Ray
• Au Oscar Chi Lim
• Origin Res Wireless Inc

Inventors

• Ozturk Muhammed Zahid
• Wu Chenshu
• Wang Beibei
• Liu K J Ray
• Au Oscar Chi-Lim

Classifications

• G01S5/0226

  Transmitters · CPC title

• G01S5/0252

  Radio frequency fingerprinting · CPC title

• G01S5/0273

  using multipath or indirect path propagation signals in position determination · CPC title

• H04L27/2647

  Arrangements specific to the receiver only (equalisation H04L27/01) · CPC title

• H04L5/0048

  Allocation of pilot signals, i.e. of signals known to the receiver (allocation of control signalling H04L5/0053; use of control signalling H04L5/0091) · CPC title