Robust radar-based gesture-recognition by user equipment

Patent metadata

Field	Value
Publication number	US-11868537-B2
Application number	US-202217661494-A
Country	US
Kind code	B2
Filing date	Apr 29, 2022
Priority date	Jul 26, 2019
Publication date	Jan 9, 2024
Grant date	Jan 9, 2024

Abstract

Systems and techniques are described for robust radar-based gesture-recognition. A radar system detects radar-based gestures on behalf of application subscribers. A state machine transitions between multiple states based on inertial sensor data. A no-gating state enables the radar system to output radar-based gestures to application subscribers. The state machine also includes a soft-gating state that prevents the radar system from outputting the radar-based gestures to the application subscribers. A hard-gating state prevents the radar system from detecting radar-based gestures altogether. The techniques and systems enable the radar system to determine when not to perform gesture-recognition, enabling user equipment to automatically reconfigure the radar system to meet user demand. By so doing, the techniques conserve power, improve accuracy, or reduce latency relative to many common techniques and systems for radar-based gesture-recognition.

First claim

What is claimed is:

1. An apparatus comprising:
a radar system that detects radar-based gestures on behalf of application subscribers;
an inertial measurement unit that produces inertial data; and
a state machine configured to transition between multiple states for controlling the radar system based on the inertial data and one or more context-sensitive transition functions, the multiple states including:
a no-gating state in which the state machine enables the radar system to output indications of the radar-based gestures to the application subscribers; and
a soft-gating state in which the radar system is prevented from outputting the indications of the radar-based gestures to one or more of the application subscribers by shielding a gesture-recognition model from radar data collected by the radar system.

2. The apparatus of claim 1 , wherein the state machine further includes a hard-gating state in which the state machine prevents the radar system from detecting the radar-based gestures.

3. The apparatus of claim 2 , wherein the hard-gating state further includes operating the radar system in a low-resolution mode that generates low-resolution radar data that is effective to disable the gesture-recognition model.

4. The apparatus of claim 2 , wherein the soft-gating state requires the radar system to utilize more power than the hard-gating state.

5. The apparatus of claim 1 , further comprising:
one or more sensors that receive sensor data; and
a machine learned activity classifier that uses machine learning techniques to recognize patterns or signatures in the sensor data to determine a device context,
wherein the transitions between the multiple states for controlling the radar system are based on the inertial data and the determined device context.

6. The apparatus of claim 5 , wherein the one or more sensors include the inertial measurement unit and the sensor data includes the inertial data.

7. The apparatus of claim 5 , wherein the one or more sensors include at least one of: an ambient light sensor, a barometer, a location sensor, an optical sensor, an infrared sensor, an accelerometer, a gyroscope, a magnetometer, a compass, or a temperature sensor.

8. The apparatus of claim 5 , wherein the device context is based on at least one of: a determination that the apparatus is being held by a user, a determination that the apparatus is moving, a determination that the apparatus is stowed, a determination that the apparatus is oriented in a particular way, a determination that the apparatus is in a particular location, or a determination that at least one of the one or more sensors is occluded.

9. The apparatus of claim 5 , wherein the device context includes at least one of: a walking context, a cycling context, a driving context, a riding context, a seated context, a stationary context, an unused context, or a stowed context.

10. The apparatus of claim 1 , wherein the soft-gating state prevents the radar system from outputting the indications of the radar-based gestures to a first set of the application subscribers and a second set of the application subscribers.

11. The apparatus of claim 10 , wherein the first set of the application subscribers and the second set of the application subscribers are determined based on one or more device contexts.

12. The apparatus of claim 1 , wherein the soft-gating state enables indications of the radar-based gestures to be used for a lower-level support function of the apparatus.

13. The apparatus of claim 12 , wherein the lower-level support function includes a system service of the apparatus or a hidden function of the apparatus.

14. A method of controlling a radar system of a device, the method performed by the device and comprising:
receiving inertial data from an inertial measurement unit of the device; and
configuring, based on the inertial data and via at least one or more context-sensitive transition functions, the device to one of multiple radar system gating states, the radar system detecting radar-based gestures on behalf of application subscribers, the multiple states including:
a no-gating radar system gating state in which the radar system is enabled to output indications of the radar-based gestures to the application subscribers; and
a soft-gating state in which the radar system is prevented from outputting the indications of the radar-based gestures to one or more of the application subscribers by shielding a gesture-recognition model from radar data collected by the radar system.

15. The method of claim 14 , wherein the multiple states further include a hard-gating state in which the radar system is prevented from detecting the radar-based gestures.

16. The method of claim 14 , wherein the soft-gating state prevents the radar system from outputting the indications of the radar-based gestures to a first set of the application subscribers and a second set of the application subscribers.

17. The method of claim 16 , wherein the first set of the application subscribers and the second set of the application subscribers are determined based on one or more device contexts.

18. The method of claim 14 , wherein the soft-gating state enables indications of the radar-based gestures to be used for a lower-level support function of the device.

19. The method of claim 18 , wherein the lower-level support function includes a system service of the device or a hidden function of the device.

20. A non-transitory computer-readable storage media storing computer-executable instructions that, when executed by at least one processor, cause the at least one processor to:
receive inertial data from an inertial measurement unit; and
configure, based on the inertial data and via at least one or more context-sensitive transition functions, a device to one of multiple radar system gating states, a radar system of the device detecting radar-based gestures on behalf of application subscribers, the multiple states including:
a no-gating radar system gating state in which the radar system is enabled to output indications of the radar-based gestures to the application subscribers; and
a soft-gating state in which the radar system is prevented from outputting the indications of the radar-based gestures to one or more of the application subscribers by shielding a gesture-recognition model from radar data collected by the radar system.

Assignees

• Google Llc

Inventors

• Hong Jung Ook
• Amihood Patrick M
• Jacobs John David
• Mengistu Abel Seleshi
• Giusti Leonardo
• Sachidanandam Vignesh
• O'Reilley Stern Devon James
• Poupyrev Ivan
• Barbello Brandon
• Kugler Tyler Reed
• Prag Johan
• Tsurkan Artur
• Chandel Alok
• Moreira Costa Lucas Dupin
• Cinek Selim Flavio

Classifications

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06V40/28

  Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113) · CPC title

• G06F2218/12

  Classification; Matching · CPC title

• G06F3/012

  Head tracking input arrangements · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title