Gesture detection in interspersed radar and network traffic signals

Patent metadata

Field	Value
Publication number	US-11487363-B2
Application number	US-202016935576-A
Country	US
Kind code	B2
Filing date	Jul 22, 2020
Priority date	Jul 29, 2019
Publication date	Nov 1, 2022
Grant date	Nov 1, 2022

Abstract

Techniques for performing gesture recognition with an electronic device are disclosed where the electronic device has a wireless communications capability using beamforming techniques and includes a plurality of millimeter wave antenna modules, each antenna module including at least one transmit antenna and at least one receive antenna, the antennas being operable in one or more frequency ranges greater than 20 GHz. Performing gesture recognition includes: simultaneous operation of the at least one transmit antenna and the at least one receive antenna so as to provide a radar capability; and detecting a presence and motion of a reflective object by analyzing magnitude and phase of signals received by the at least one receive antenna and resulting from reflection of signals transmitted by the transmit antenna and reflected by the reflective object.

First claim

What is claimed is:

1. A method of gesture recognition comprising:
performing gesture recognition and network data communications using an electronic device, the electronic device having both a radar capability and a wireless communications capability based on millimeter wave signals, the electronic device including at least one transmit antenna and at least one receive antenna that are operable in one or more frequency ranges greater than 20 GHz,
wherein the performing gesture recognition includes:
simultaneously operating of the at least one transmit antenna and the at least one receive antenna so as to provide the radar capability, including receiving millimeter wave signals using the at least one receive antenna during a first portion of a time frame;
detecting a presence and a motion of a reflective object by analyzing at least phase of millimeter wave signals received by the at least one receive antenna and resultant from reflection of signals transmitted by the at least one transmit antenna by the reflective object; and
outputting a recognized gesture,

wherein the performing network data communications includes:
receiving millimeter wave signals using the at least one receive antenna during a second portion of the time frame; and
demodulating and decoding the millimeter wave signals received during the second portion of the time frame to generate data bits,

wherein the first portion of the time frame comprises a first plurality of non-contiguous time periods for receiving signals for gesture recognition,
wherein the second portion of the time frame comprises a second plurality of non-contiguous time periods for receiving signals for network data communications, and
wherein the first plurality of non-contiguous time periods of the time frame is interspersed with the second plurality of non-contiguous time periods of the time frame.

2. The method of claim 1 , wherein millimeter wave signals received during at least one of the first plurality of non-contiguous time periods comprises a burst of pulses.

3. The method of claim 2 , wherein the analyzing at least phase of the millimeter wave signals comprises combining multiple pulses in the burst of pulses to generate a combined channel impulse response (CIR).

4. The method of claim 3 , where the analyzing at least phase of the millimeter wave signals further comprises a frequency-based analysis, including:
applying a transform operation to the combined CIR to generate a spectrogram;
providing the spectrogram to a trained machine learning (ML) classifier; and
obtaining the recognized gesture from the trained ML classifier.

5. The method of claim 4 , wherein the applying the transform operation comprises performing a Fast Fourier Transform (FFT) operation on the CIR to generate the spectrogram.

6. The method of claim 3 , where the analyzing at least phase of millimeter wave signals further comprises a time-based analysis, including:
generating an estimated slope reflecting a rate of change of phase based on the combined CIR;
comparing the estimated slope to a threshold; and
determining a pattern of slope polarities based on the comparing the estimated slope to the threshold; and
generating the recognized gesture based on the pattern of the slope polarities.

7. The method of claim 1 , wherein the performing gesture recognition includes recognizing a double tap gesture.

8. The method of claim 1 , wherein the at least one transmit antenna and at least one receive antenna are compatible with one or both of IEEE 802.11ad and IEEE 802.11ay wi-fi protocols.

9. The method of claim 1 , wherein each of the at least one transmit antenna and the at least one receive comprises a plurality of antenna elements.

10. The method of claim 1 , wherein the reflective object is one or more of a hand or other appendage of a user, or a hand held object.

11. The method of claim 1 , wherein the signals transmitted by the at least one transmit antenna include two complementary Golay sequences used as two sequential radar pulses.

12. The method of claim 1 , wherein one or both of the at least one transmit antenna and the at least one receive antenna are operable in a 60 GHz band.

13. The method of claim 1 , further comprising executing a graphical user interface (GUI) operation, responsive to the recognized gesture.

14. An apparatus comprising:
a processor and an electronic device having both a radar capability and a wireless communications capability based on millimeter wave signals, the electronic device including at least one transmit antenna and at least one receive antenna that are operable in one or more frequency ranges greater than 20 GHz;
wherein the processor is configured to perform gesture recognition with the electronic device by:
simultaneously operating the at least one transmit antenna and the at least one receive antenna so as to provide the radar capability, including receiving millimeter wave signals using the at least one receive antenna during a first portion of a time frame;
detecting a presence and a motion of a reflective object by analyzing at least phase of millimeter wave signals received by the at least one receive antenna and resultant from reflection of signals transmitted by the at least one transmit antenna and reflected by the reflective object; and
outputting a recognized gesture,

wherein the processor is further configured to perform network data communications with the electronic device by:
receiving millimeter wave signals using the at least one receive antenna during a second portion of the time frame; and
demodulating and decoding the millimeter wave signals received during the second portion of the time frame to generate data bits,

wherein the first portion of the time frame comprises a first plurality of non-contiguous time periods for receiving signals for gesture recognition,
wherein the second portion of the time frame comprises a second plurality of non-contiguous time periods for receiving signals for network data communications, and
wherein the first plurality of non-contiguous time periods of the time frame is interspersed with the second plurality of non-contiguous time periods of the time frame.

15. The apparatus of claim 14 , wherein millimeter wave signals received during at least one of the first plurality of non-contiguous time periods comprises a burst of pulses.

16. The apparatus of claim 15 , wherein the analyzing at least phase of millimeter wave signals comprises combining multiple pulses in the burst of pulses to generate a combined channel impulse response (CIR).

17. The apparatus of claim 16 , where the analyzing at least phase of millimeter wave signals further comprises a frequency-based analysis, including:
applying a transform operation to the combined CIR to generate a spectrogram;
providing the spectrogram to a trained machine learning (ML) classifier; and
obtaining the recognized gesture from the trained ML classifier.

18. The apparatus of claim 17 , wherein the applying the transform operation comprises performing a Fast Fourier Transform (FFT) operation on the CIR to generate the spectrogram.

19. The apparatus of claim 16 , where the analyzing at least phase of millimeter wave signals further comprises a time-based analysis, including:
generating an estimated slope reflecting a rate of change of phase based on the combined CIR;
comparing the estimated slope to a threshold; and
determining a pattern of slope polarities based on the comparing the estimated slope to the threshold; and
generating th

Assignees

• Qualcomm Inc

Inventors

• Hof Eran
• Sanderovich Amichai
• Levitan Evgeny
• Sagi Ariel
• Hemo Evyatar
• Sambhwani Sharad

Classifications

• H01Q1/3233

  particular used as part of a sensor or in a security system, e.g. for automotive radar, navigation systems · CPC title

• G01S7/415

  Identification of targets based on measurements of movement associated with the target · CPC title

• G06F2203/04101

  2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup · CPC title

• G06F3/0416

  Control or interface arrangements specially adapted for digitisers · CPC title

• H04B7/043

  using best eigenmode, e.g. beam forming or beam steering · CPC title