Radar-enabled sensor fusion

Patent metadata

Field	Value
Publication number	US-10401490-B2
Application number	US-201615286512-A
Country	US
Kind code	B2
Filing date	Oct 5, 2016
Priority date	Oct 6, 2015
Publication date	Sep 3, 2019
Grant date	Sep 3, 2019

Abstract

This document describes apparatuses and techniques for radar-enabled sensor fusion. In some aspects, a radar field is provided and reflection signals that correspond to a target in the radar field are received. The reflection signals are transformed to provide radar data, from which a radar feature indicating a physical characteristic of the target is extracted. Based on the radar features, a sensor is activated to provide supplemental sensor data associated with the physical characteristic. The radar feature is then augmented with the supplemental sensor data to enhance the radar feature, such as by increasing an accuracy or resolution of the radar feature. By so doing, performance of sensor-based applications, which rely on the enhanced radar features, can be improved.

First claim

We claim:

1. A computer-implemented method comprising:
providing, via a radar-emitting element of a device, a radar field;
receiving, via a radar-receiving element of the device, one or more reflection signals that correspond to a target in the radar field;
transforming the one or more reflection signals to provide radar data;
extracting, from the radar data, a radar feature comprising an indication of a shape of the target;
activating, based on the radar feature, a sensor of the device to provide supplemental sensor data;
augmenting the radar feature with the supplemental sensor data to enhance the radar feature; and
providing the enhanced radar feature to a sensor-based application effective to improve performance of the sensor-based application.

2. The computer-implemented method of claim 1 , wherein the radar feature further comprises an indication of target detection, target reflection, target motion, or target position.

3. The computer-implemented method of claim 2 , wherein the radar feature further comprises an indication of a number of targets, total reflected energy, moving energy, one-dimensional velocity, one-dimensional velocity dispersion, three-dimensional spatial coordinates, or one-dimensional spatial dispersion.

4. The computer-implemented method of claim 1 , wherein transforming the one or more reflection signals comprises performing one of a range-Doppler transform, range profile transform, micro-Doppler transform, I/Q transform, or spectrogram transform.

5. The computer-implemented method of claim 1 , wherein the sensor-based application comprises proximity detection, user detection, user tracking, activity detection, facial recognition, breathing detection, or motion signature identification.

6. The computer-implemented method of claim 1 , wherein the sensor that provides the supplemental sensor data comprises an accelerometer, gyroscope, hall effect sensor, magnetometer, temperature sensor, microphone, capacitive sensor, proximity sensor, ambient light sensor, red-green-blue (RGB) image sensor, infrared sensor, or depth sensor.

7. The computer-implemented method of claim 1 , wherein augmenting the radar feature with the supplemental sensor data is effective to enhance the radar feature by:
increasing a positional accuracy of the radar feature;
mitigating a false positive detection attributed to the radar feature;
increasing a spatial resolution of the radar feature;
increasing a surface resolution of the radar feature; or
improving a classification precision of the radar feature.

8. The computer-implemented method of claim 1 , wherein a radar sensor of the device includes the radar-emitting element and radar-receiving element, the radar sensor consumes less power than the sensor that is activated, and the radar sensor is operated while the sensor is in a low-power state until a change in the one or more reflection signals is detected is effective to conserve power of the device.

9. An apparatus comprising:
one or more computer processors;
a radar sensor comprising:
a radar-emitting element configured to provide radar fields; and
a radar-receiving element configured to receive reflection signals that correspond to the radar fields;

a plurality of sensors, each of the sensors comprising a distinct type of sensor configured to sense a respective environmental variance of the apparatus; and
one or more computer-readable storage media having instructions stored thereon that, responsive to execution by the one or more computer processors, implement a sensor fusion engine to:
provide, via the radar-emitting element, a radar field;
receive, via the radar-receiving element, one or more reflection signals that correspond to a target in the radar field;
transform the one or more reflection signals to provide radar data;
abstract, from the radar data, a radar feature that indicates a physical characteristic of the target;
select, based on the radar feature being a first type of radar feature, a first sensor of the plurality of sensors; or
select, based on the radar feature being a second type of radar feature, a second sensor of the plurality of sensors that is a different type of sensor than the first sensor;
activate the selected sensor and receive supplemental sensor data from the selected activated sensor, the supplemental sensor data describing an environmental variance corresponding to the distinct type of the activated sensor that further indicates the physical characteristic of the target;
augment the radar feature with the supplemental sensor data; and
provide the augmented radar feature to a sensor-based application.

10. The apparatus of claim 9 , wherein the first or second type of the radar feature comprises a surface feature, motion feature, position feature, or detection feature.

11. The apparatus of claim 9 , wherein the first or second sensor comprises an accelerometer, gyroscope, hall effect sensor, magnetometer, temperature sensor, microphone, capacitive sensor, proximity sensor, ambient light sensor, red-green-blue (RGB) image sensor, infrared sensor, or depth sensor.

12. The apparatus of claim 9 , wherein the radar feature indicates a number of targets in the radar field, total reflected energy, moving energy, one-dimensional velocity, one-dimensional velocity dispersion, three-dimensional spatial coordinates, or one-dimensional spatial dispersion.

13. The apparatus of claim 9 , wherein the radar sensor further comprises an antenna array through which the radar-emitting element provides the radar field and the radar-receiving element receives the one or more reflection signals.

14. The apparatus of claim 13 , wherein the antenna array comprises at least two antennas coupled directly to the radar-emitting element and at least two antennas coupled directly to the radar-receiving element.

15. The apparatus of claim 9 , wherein the apparatus is embodied as a smart-phone, smart-glasses, smart-watch, tablet computer, laptop computer, set-top box, smart-appliance, home automation controller, or television.

16. A system-on-chip comprising:
a radar-emitting element configured to provide radar fields;
a radar-receiving element configured to receive reflection signals;
an antenna array through which the radar field is provided and through which the reflection signals are received;
a data interface configured to enable communication with one or more sensors;
a processor; and
a computer-readable storage media having instructions stored thereon that, responsive to execution by the computer processor, implement a sensor fusion engine to:
cause the radar-emitting element to provide a radar field;
receive, via the radar-receiving element, one or more reflection signals that correspond to the radar field;
transform the one or more reflection signals to provide radar data;
resolve the radar data into a radar feature comprising a shape of an object causing the one or more reflection signals;
activate, based on the radar feature, the one or more sensors;
receive, from the one or more sensors, sensor data that indicates an environmental variance;
augment the radar feature with the sensor data that indicates the environmental variance to provide an enhanced radar feature; and
expose the enhanced radar feature to a sensor-based application effective to improve performance of the sensor-based application.

17. The system-on-chip of claim 16 , wherein the radar feature further comprises one of target detection, target reflection, target motion, or target position.

18. The system-on-chip of claim

Assignees

• Google Inc
• Google Llc

Inventors

• Gillian Nicholas Edward
• Schwesig Carsten C
• Lien Jaime
• Amihood Patrick M
• Poupyrev Ivan

Classifications

• G01S7/415 Primary

  Identification of targets based on measurements of movement associated with the target · CPC title

• G06V10/806

  of extracted features · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F18/41

  Interactive pattern learning with a human teacher · CPC title

• G06F18/253

  of extracted features · CPC title