Devices, systems, and methods for detecting gestures using wireless communication signals

Patent metadata

Field	Value
Publication number	US-9971414-B2
Application number	US-201414779946-A
Country	US
Kind code	B2
Filing date	Apr 1, 2014
Priority date	Apr 1, 2013
Publication date	May 15, 2018
Grant date	May 15, 2018

Abstract

Examples of systems, devices, and methods are described herein that can provide for gesture recognition. Wireless communication signals are received from sources in an environment (e.g. cellular telephones, computers, etc.). Features of the wireless communication signals (e.g. Doppler shifts) are extracted and utilized to identify gestures. The use of wireless communication signals accordingly make possible gesture recognition in a whole-home environment that identifies gestures performed through walls or other obstacles.

First claim

What is claimed is:

1. A device comprising:
a receiver configured to receive wireless communication signals, each wireless communication signal received from a plurality of sub-channels;
at least one processing unit configured to identify features in the wireless communication signals, wherein identifying the features in the wireless communication signals comprises:
equalizing received symbols in each sub-channel of the plurality of sub-channels, wherein equalizing the received symbols comprises multiplying each received symbol in a respective sub-channel with a ratio including a first received symbol of the respective sub-channel to provide equalized symbols;
transforming the equalized symbols with a Fast Fourier Transform (FFT) to provide a narrowband signal; and
analyzing a Doppler profile of the narrowband signal based on the equalized symbols to identify the features in the wireless communication signals;
a memory accessible to the at least one processing unit and storing information regarding features in wireless communication signals caused by gestures; and
wherein the at least one processing unit is further configured to identify a gesture based on the identified features in the wireless communication signals.

2. The device of claim 1 , wherein the gestures include two-dimensional and three-dimensional gestures.

3. The device of claim 1 , wherein the wireless communication signals include Orthogonal Frequency-Division Multiplexing (OFDM) signals, wherein the received symbols correspond to OFDM symbols.

4. The device of claim 1 , wherein the wireless communication signals includes received signal strength indicators (RSSI), phase information, or combinations thereof.

5. The device of claim 1 , wherein the wireless communication signals are transmitted from a cellular telephone, computer, or wireless router.

6. The device of claim 1 , wherein identifying features in the wireless communication signals further comprises extracting a plurality of Doppler shifts in the Doppler profile.

7. The device of claim 6 , wherein identifying features in the wireless communication signals further comprises:
identifying a set of segments in the plurality of Doppler shifts; and
classifying the set of segments as either positive Doppler shifts, negative Doppler shifts, or positive/negative Doppler shifts.

8. The device of claim 7 , wherein classifying the set of segments as either positive Doppler shifts, negative Doppler shifts, or positive/negative Doppler shifts comprises:
correlating the set of segments with a gesture signature, the gesture signature indicative of a known set of segments.

9. The device of claim 1 wherein the wireless communication signals comprise beacon signals.

10. The device of claim 1 wherein the at least one processing unit is configured to identify a preamble gesture from the features in the communication signals, and wherein the at least one processing unit is further configured to identify when a subsequent gesture is made by a same entity performing the preamble gesture.

11. The device of claim 1 , further comprising a transmitter configured to transmit a control signal responsive to identification of the gesture.

12. The device of claim 1 , wherein equalizing the received symbols in each sub-channel of the plurality of sub-channels according to the first received symbol of each respective sub-channel comprises modifying each received symbol using the respective first received symbol of each respective sub-channel.

13. A method comprising:
receiving wireless communication signals from devices in an environment, each wireless communication signal received from a plurality of sub-channels;
detecting features in the wireless communication signals, wherein detecting the features in the wireless communication signals comprises:
equalizing received symbols in each sub-channel of the plurality of sub-channels, wherein equalizing the received symbols comprises multiplying each received symbol in a respective sub-channel with a ratio including a first received symbol of the respective sub-channel to provide equalized symbols;
transforming the equalized symbols with a Fast Fourier Transform (FFT) to provide a narrowband signal; and
analyzing a Doppler profile of the narrowband signal based on the equalized symbols to identify the features in the wireless communication signals;

identifying a gesture based, at least in part, on the features in the wireless communication signals; and
providing a control signal responsive to identification of the gesture.

14. The method of claim 13 , wherein the wireless communication signals are Orthogonal Frequency Division Multiplexing (OFDM) signals.

15. The method of claim 13 , wherein said detecting features in the wireless communication signals further comprises identifying at least one Doppler shift in the Doppler profile of the narrowband signal.

16. The method of claim 15 , wherein identifying a gesture based, at least in part, on the features in the wireless communication signals comprises comparing the features to a plurality of stored gesture signatures.

17. The method of claim 13 , wherein said identifying a gesture comprises correlating the features with a database of stored features related to gestures.

18. The method of claim 17 , wherein said correlating comprises using machine learning, pattern matching, or combinations thereof.

19. The method of claim 13 , wherein the wireless communication signals are provided by a cellular telephone, computer, router, or combinations thereof.

20. A method, comprising:
receiving wireless communication signals from devices in an environment;
detecting features in the wireless communication signals;
identifying a gesture based, at least in part, on the features in the wireless communication signals, wherein the gesture comprises performance of a preamble gesture, wherein identifying the gesture based, at least in part, on the features in the wireless communication signals comprises:
equalizing received symbols in each sub-channel of the plurality of sub-channels to provide equalized symbols, wherein equalizing the received symbols comprises first received symbol of the respective sub-channel to provide equalized symbols;
transforming the equalized symbols with a Fast Fourier Transform (FFT) to provide a narrowband signal; and
comparing a Doppler profile of the narrowband signal with the plurality of gesture signatures to identify the gesture;
determining that a Multiple-Input Multiple-Output (MIMO) channel of a plurality of MIMO channels maximizes Doppler energy from an entity associated with the preamble gesture;
tracking the entity associated with the preamble gesture with the MIMO channel;
identifying a subsequent gesture based, at least in part, on the features in additional wireless communication signals from the tracked entity associated with the preamble gesture; and
providing a control signal responsive to identification of the subsequent gesture.

21. The method of claim 20 , further comprising detecting further features in the wireless communication signals and determining whether the further features occurred due to an activity of the entity.

22. A system comprising:
a wireless communications device configured to provide periodic beacon signals using an Orthogonal Frequency Division Multiplexing (OFDM) protocol, each periodic beacon signal received from a plurality of sub-channels;
a router configured to receive the periodic beacon signals f

Assignees

• Univ Washington Through Its Center For Commercialization

Inventors

• Gollakota Shyamnath
• Patel Shwetak N
• Pu Qifan
• Gupta Sidhant

Classifications

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F13/28 Primary

  using burst mode transfer, e.g. direct memory access {DMA}, cycle steal (G06F13/32 takes precedence) · CPC title