Method, apparatus, and system for wirelessly tracking keystrokes

Patent metadata

Field	Value
Publication number	US-2021215788-A1
Application number	US-202117214836-A
Country	US
Kind code	A1
Filing date	Mar 27, 2021
Priority date	Jul 17, 2015
Publication date	Jul 15, 2021
Grant date	—

Abstract

Methods, apparatus and systems for wirelessly tracking keystrokes on a surface are described. In one example, a described system comprises: a transmitter configured for transmitting, using a transmit antenna, a first wireless signal through a wireless channel of a venue including the surface; a receiver configured for receiving, using a plurality of receive antennas, a second wireless signal through the wireless channel, wherein the second wireless signal comprises a reflection of the first wireless signal by at least one moving object in the venue; and a processor. The processor is configured for: obtaining, regarding each receive antenna, a time series of channel information (CI) of the wireless channel based on the second wireless signal, detecting at least one keystroke on the surface based on the time series of CI (TSCI) obtained regarding each receive antenna, determining at least one location of the at least one keystroke on the surface, and determining at least one key associated with the at least one keystroke based on the at least one location.

First claim

We claim:

1 . A system for wirelessly tracking keystrokes on a surface, comprising:
a transmitter configured for transmitting, using a transmit antenna, a first wireless signal through a wireless channel of a venue including the surface; a receiver configured for receiving, using a plurality of receive antennas, a second wireless signal through the wireless channel, wherein the second wireless signal comprises a reflection of the first wireless signal by at least one moving object in the venue; and a processor configured for:
obtaining, regarding each of the plurality of receive antennas, a time series of channel information (CI) of the wireless channel based on the second wireless signal, wherein each CI comprises at least one of: a channel state information (CSI), channel impulse response (CIR), channel frequency response (CFR), or received signal strength index (RSSI),
detecting at least one keystroke on the surface based on the time series of CI (TSCI) obtained regarding each of the plurality of receive antennas,
determining at least one location of the at least one keystroke on the surface, and
determining at least one key associated with the at least one keystroke based on the at least one location.

2 . The system of claim 1 , wherein:
each CI comprises a CIR; the first wireless signal is carried on a millimeter wave; each of the at least one moving object is a finger or toe configured to type on a virtual keyboard to perform keystrokes, and has a location determined based on a plurality of spatial bins in the venue; each of the plurality of spatial bins is determined by: a respective angle direction and a respective distance range originating from the receiver; and each angle direction is identified by a corresponding azimuth angle and a corresponding elevation angle.

3 . The system of claim 2 , wherein detecting the at least one keystroke comprises:
detecting a motion of each of the at least one moving object on the surface based on the time series of CI (TSCI) obtained regarding each of the plurality of receive antennas; and recognizing the motion as a keystroke performed by the moving object.

4 . The system of claim 3 , wherein detecting the motion of each moving object comprises:
for each time instance, for each of a plurality of distance ranges of interest and for each of the plurality of receive antennas,
computing, for the distance range of interest, a differential CIR based on a CIR amplitude measured by the receive antenna at the time instance and a CIR amplitude measured by the receive antenna at a preceding time instance before the time instance;
computing, for the receive antenna and the distance range of interest, a measurement score based on: the differential CIR, a sample mean of a reference differential CIR, and a standard deviation of the reference differential CIR;
comparing the measurement score with a first threshold; and
determining that a candidate motion is detected by the receive antenna at the distance range of interest when the measurement score is larger than the first threshold.

5 . The system of claim 4 , wherein:
the reference differential CIR is computed based on amplitudes of a reference CIR measured by the receive antenna at two consecutive time instances; and the reference CIR is a CIR of the wireless channel and is obtained with no moving object in the venue.

6 . The system of claim 4 , wherein detecting the motion of each moving object further comprises:
for each candidate motion, determining that the candidate motion is a target motion when the candidate motion is detected by a majority of the plurality of receive antennas at a time instance.

7 . The system of claim 6 , wherein detecting the motion of each moving object further comprises:
for each target motion,
determining a start time of the target motion based on a first time instance when a majority of the plurality of receive antennas detect the target motion; and
determining an end time of the target motion based on a last time instance when a majority of the plurality of receive antennas detect the target motion.

8 . The system of claim 7 , wherein detecting the motion of each moving object further comprises:
for each target motion, determining a target distance range of the target motion based on a distance range that maximizes a summation of measurement scores computed for all of the plurality of receive antennas and for all time instances from the start time to the end time.

9 . The system of claim 8 , wherein recognizing the motion as a keystroke comprises:
for each target motion, for each of the plurality of receive antennas, and for each of three spatially consecutive distance ranges centered at the target distance range of the target motion,
computing, a peak height of a CIR phase measured by the receive antenna for the distance range;
computing a first total power of non-zero frequency components of a CIR signal measured by the receive antenna for the distance range; and
computing a second total power of all frequency components of the CIR signal measured by the receive antenna for the distance range.

10 . The system of claim 9 , wherein recognizing the motion as a keystroke further comprises:
for each target motion,
computing an aggregated peak height based on peak heights of CIR phases measured by all of the plurality of receive antennas for all of the three spatially consecutive distance ranges;
comparing the aggregated peak height with a second threshold;
computing a first aggregated power based on first total powers computed for all of the plurality of receive antennas and for all of the three spatially consecutive distance ranges;
computing a second aggregated power based on second total powers computed for all of the plurality of receive antennas and for all of the three spatially consecutive distance ranges;
computing a dynamic level, which represents a reflection area associated with the target motion, based on a ratio between the first aggregated power and the second aggregated power;
comparing the dynamic level with a third threshold; and
recognizing the target motion as a keystroke motion, when:
the aggregated peak height is smaller than the second threshold, and
the dynamic level is smaller than the third threshold.

11 . The system of claim 10 , wherein:
each of the second threshold and the third threshold is predetermined based on a calibration of the system; the calibration is related to:
keystrokes each corresponding to a known target distance range, and
hand motions each corresponding to a shift from one keystroke to another;
the second threshold is computed by averaging peak heights of CIR phases of keystrokes and hand motions at the target distance range; and the third threshold is computed by averaging dynamic levels of keystrokes and hand motions at the target distance range.

12 . The system of claim 9 , wherein recognizing the motion as a keystroke further comprises:
for each target motion and for each of a plurality of antenna-distance groups, wherein each antenna-distance group includes: a respective one of the plurality of receive antennas and a respective one of three spatially consecutive distance ranges centered at the target distance range of the target motion,
comparing the peak height with a second threshold,
computing a dynamic level, which represents a reflection area associated with the target motion, based on a ratio between the first total power and the second total power,
comparing the dynamic level with a third threshold, wherein each of the second threshold and the third threshold is predetermined based on a calibration of t

Assignees

• Hu Yuqian
• Wang Beibei
• Wu Chenshu
• Liu K J Ray
• Au Oscar Chi Lim

Inventors

• Hu Yuqian
• Wang Beibei
• Wu Chenshu
• Liu K J Ray
• Au Oscar Chi-Lim

Classifications

• G01S13/88

  Radar or analogous systems specially adapted for specific applications (electromagnetic prospecting or detecting of objects, e.g. near-field detection, G01V3/00) · CPC title

• G01S13/003

  Bistatic radar systems; Multistatic radar systems · CPC title

• G01S13/42

  Simultaneous measurement of distance and other co-ordinates (indirect measurement G01S13/46) · CPC title

• G01S7/415

  Identification of targets based on measurements of movement associated with the target · CPC title

• G01S13/52

  Discriminating between fixed and moving objects or between objects moving at different speeds · CPC title