Method, apparatus, and system for people counting and recognition based on rhythmic motion monitoring

Patent metadata

Field	Value
Publication number	US-12046040-B2
Application number	US-202016871004-A
Country	US
Kind code	B2
Filing date	May 10, 2020
Priority date	Jul 17, 2015
Publication date	Jul 23, 2024
Grant date	Jul 23, 2024

Abstract

Methods, apparatus and systems for counting and recognizing objects or people based on rhythmic motion monitoring are described. A described method comprises: obtaining N1 time series of channel information (TSCI) of a wireless multipath channel that is impacted by a rhythmic motion of an object in a venue, wherein the N1 TSCI is extracted from a wireless signal transmitted from a transmitter to a receiver through the wireless multipath channel; decomposing each of the N1 TSCI into N2 time series of channel information components (TSCIC), wherein a channel information component (CIC) of each of the N2 TSCIC at a time comprises a respective component of a channel information (CI) of the TSCI at the time, wherein N1 and N2 are positive integers; monitoring the rhythmic motion of the object based on at least one of: the N1*N2 TSCIC and the N1 TSCI.

First claim

We claim:

1. A system for rhythmic motion monitoring, comprising:
a transmitter configured for transmitting a wireless signal through a wireless multipath channel that is impacted by a rhythmic motion of an object in a venue;
a receiver configured for:
receiving the wireless signal through the wireless multipath channel, and
extracting N1 time series of channel information (TSCI) of the wireless multipath channel from the wireless signal modulated by the wireless multipath channel that is impacted by the rhythmic motion of the object, wherein:
there are N1 links between the transmitter and the receiver, each link associated with a respective transmit antenna of the transmitter and a respective receive antenna of the receiver,
each TSCI is associated with a respective link,
each channel information (CI) comprises one of: channel state information (CSI), channel frequency response (CFR), or channel impulse response (CIR),
each CI has N2 CI components (CIC),
each CIC comprises one of: a component of a CSI, a subcarrier of a CFR, or a tap of a CIR; and

a processor configured for:
decomposing each of the N1 TSCI into N2 time series of channel information components (TSCIC) to obtain N1*N2 TSCIC, wherein there are N1*N2 combinations of link and CIC associated with the N1*N2 TSCIC, wherein N1 and N2 are positive integers,
computing N1*N2 short-time frequency transformation (STFT) in a sliding time window, wherein each of the N1*N2 STFT is an STFT of the CIC of a respective TSCIC in the sliding time window, wherein each of the N1*N2 STFT has a same amount N3 of frequency components, wherein:
each combination (m,k) of m{circumflex over ( )}{th} link and k{circumflex over ( )}{th} CIC is associated with a respective one of the N1*N2 TSCIC and with a respective one of the N1*N2 STFT,
the STFT comprises at least one of: short-time Fourier transform, trigonometric transform, discrete cosine transform, discrete sine transform, Hadamard transform, slant transform, integer transform, power-of-2 transform, fast transform, sparse transform, wavelet transform, Laplace transform, Hilbert transform, graph-based transform, or orthogonal transform,

computing a number of intermediate STFT (ISTFT) each comprising N3 respective frequency components for the sliding time window based on respective subsets of the N1*N2 STFT, wherein:
each frequency component of each ISTFT is a respective first aggregate of the respective frequency components of the respective subset of the N1*N2 STFT,
any aggregate quantity comprises at least one of: average, weighted average, mean, trimmed mean, median, mode, arithmetic mean, geometric mean, harmonic mean, truncated mean, generalized mean, power mean, f-mean, or inter-quartile mean of individual quantities,

computing a combined STFT (CSTFT) comprising N3 frequency components for the sliding time window based on the number of ISTFT, wherein each frequency component of the CSTFT is a respective second aggregate of the respective frequency components of the number of ISTFT,
for each frequency component index q:
selecting a respective subset of the N1*N2 combinations of link and CIC, based on the q{circumflex over ( )}{th} frequency components of the N1*N2 STFT and a respective q{circumflex over ( )}{th} selection criterion, wherein each selected combination of link and CIC is associated with a respective selected TSCIC and a respective selected STFT,
computing the q{circumflex over ( )}{th} frequency component of each ISTFT as a respective first aggregate of the q{circumflex over ( )}{th} frequency component of all the selected STFT associated with the ISTFT, and
computing the q{circumflex over ( )}{th} frequency component of the CSTFT as the respective second aggregate of the q{circumflex over ( )}{th} frequency components of the number of ISTFT,

monitoring the rhythmic motion of the object based on the CSTFT, and
triggering a response action based on the monitoring of the rhythmic motion of the object.

2. The system of claim 1 , wherein:
each CI of the N1 TSCI is associated with a time stamp;
each CI of the N1 TSCI comprises at least one of: N2 frequency-domain components, and N2 time-domain components; and
the processor is further configured for performing at least one of the following:
transforming, by a frequency transform (FT), each CI with N2 time-domain components to another CI with N2 frequency-domain components,
transforming, by an inverse frequency transform (IFT), each CI with N2 frequency-domain components to another CI with N2 time-domain components, and
uniformly re-sampling each CI with time stamps evenly spaced in time.

3. The system of claim 1 , wherein the processor is further configured for:
segmenting each TSCI into overlapping segments of CI, wherein each of the overlapping segments of CI comprises CI each associated with a time stamp in a respective sliding time window; and
monitoring the rhythmic motion of the object in each of the overlapping segments of CI.

4. The system of claim 1 , wherein the processor is further configured for:
adjusting a sampling time of at least one CI of a TSCI such that all CI of the TSCI and all CIC of the N2 associated TSCIC are uniformly sampled with time stamps evenly spaced in time; and
computing the STFT of each of the N1*N2 TSCIC in each of the overlapping segments after adjusting the sampling time.

5. The system of claim 4 , wherein:
the sampling time of a CI is adjusted by interpolating a re-sampled CI at a desirable time stamp based on the CI and its neighboring CI; and
the sampling time of a CIC associated with the CI is adjusted by at least one of: copying the respective component of the re-sampled CI, and interpolating a respective re-sampled CIC at the desirable time stamp based on the CIC and its neighboring CIC.

6. The system of claim 4 , wherein monitoring the rhythmic motion of the object comprises:
computing N1*N2 analytics associated with the rhythmic motion of the object based on the STFT of the N1*N2 TSCIC; and
computing a combined analytics based on a function of the N1*N2 analytics, wherein each of the N1*N2 analytics is associated with a respective one of the N1*N2 TSCIC.

7. The system of claim 1 , wherein monitoring the rhythmic motion comprises:
computing N1 ISTFT, wherein each of the N1 ISTFT is associated with one of the N1 links and is computed as a function of the STFT of respective N2 TSCIC associated with the link.

8. The system of claim 1 , wherein monitoring the rhythmic motion of the object comprises:
computing N2 ISTFT, wherein each of the N2 ISTFT is associated with one of the N2 CIC and is computed based on respective N1 TSCIC associated with the CIC; and
for each frequency component q of each of the N2 ISTFT: selecting the respective subset of the N1*N2 combinations of link and CIC, wherein the CIC of each selected combination is the respective CIC associated with the ISTFT.

9. The system of claim 8 , wherein each of the N2 ISTFT is computed as a function of the STFT of the respective N1 TSCIC in the overlapping segment.

10. The system of claim 7 , wherein monitoring the rhythmic motion of the object comprises:
computing the CSTFT based on a function of the N1 ISTFT.

11. The system of claim 1 , wherein monitoring the rhythmic motion of the object comprises computing at least one analytics associated with the rhythmic motion of the object based on a processing of at least one of: the N1 TSCI and the N1*N2 TSCIC, wherein the at least one analytics comprises at least one of:
a frequency, a period, a periodicity parameter, a rhythm, an intensity, a phase, a musical note, a pitch, an expression with a rhythm, a musical rhythm, a punctuated r

Assignees

• Wang Fengyu
• Wang Beibei
• Regani Sai Deepika
• Zhang Feng
• Wu Chenshu
• Bugos Dan
• Shih Chun Chia Jack
• Liu K J Ray
• Au Oscar Chi Lim
• Origin Res Wireless Inc

Inventors

• Wang Fengyu
• Wang Beibei
• Regani Sai Deepika
• Zhang Feng
• Wu Chenshu
• Bugos Dan
• Shih Chun-Chia Jack
• Liu K J Ray
• Au Oscar Chi-Lim

Classifications

• G06V40/10

  Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands · CPC title

• H04B7/12

  Frequency diversity · CPC title

• G06F2218/10

  by analysing the shape of a waveform, e.g. extracting parameters relating to peaks · CPC title

• H04L25/0224

  using sounding signals · CPC title

• G06V20/52 Primary

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title