Integration of tracking with classifier in mmwave radar

Patent metadata

Field	Value
Publication number	US-11435443-B2
Application number	US-201916660194-A
Country	US
Kind code	B2
Filing date	Oct 22, 2019
Priority date	Oct 22, 2019
Publication date	Sep 6, 2022
Grant date	Sep 6, 2022

Abstract

In an embodiment, a method for tracking a target using a millimeter-wave radar includes: receiving radar signals using the millimeter-wave radar; generating a range-Doppler map based on the received radar signals; detecting a target based on the range-Doppler map; tracking the target using a track; generating a predicted activity label based on the track, where the predicted activity label is indicative of an actual activity of the target; generating a Doppler spectrogram based on the track; generating a temporary activity label based on the Doppler spectrogram; assigning an uncertainty value to the temporary activity label, where the uncertainty value is indicative of a confidence level that the temporary activity label is an actual activity of the target; and generating a final activity label based on the uncertainty value.

First claim

What is claimed is:

1. A method for tracking a target using a millimeter-wave radar, the method comprising:
receiving radar signals using the millimeter-wave radar;
generating a range-Doppler map based on the received radar signals;
detecting a target based on the range-Doppler map;
tracking the target using a track;
generating a predicted activity label based on the track, wherein the predicted activity label is indicative of an actual activity of the target;
generating a Doppler spectrogram based on the track;
generating a temporary activity label based on the Doppler spectrogram;
assigning an uncertainty value to the temporary activity label, wherein the uncertainty value is indicative of a confidence level that the temporary activity label is an actual activity of the target; and
generating a final activity label based on the uncertainty value.

2. The method of claim 1 , wherein generating the final activity label comprises, when the uncertainty value is above a predetermined threshold, using the predicted activity label as the final activity label, and when the uncertainty value is below the predetermined threshold, using the temporary activity label as the final activity label.

3. The method of claim 2 , wherein generating the final activity label comprises using elliptical gating to determine whether the uncertainty value is above or below the predetermined threshold.

4. The method of claim 1 , further comprising using an activity model to generate the temporary activity label, wherein the activity model comprises a first stage and a second stage, the first stage comprising a long short-term memory (LSTM) network and the second stage comprising a fully connected layer, wherein the second stage has an input coupled to an output of the first stage.

5. The method of claim 4 , further comprising training the first stage with Doppler spectrograms snippets of truncated target activities.

6. The method of claim 4 , further comprising training the second stage with Doppler spectrograms snippets of transitions between target activities.

7. The method of claim 4 , further comprising:
training the first stage with Doppler spectrograms snippets of truncated target activities; and
after training the first stage, training the second stage with Doppler spectrograms snippets of transitions between target activities.

8. The method of claim 1 , wherein tracking the target comprises tracking the target using an unscented Kalman filter.

9. The method of claim 1 , wherein the target is a human target.

10. The method of claim 1 , wherein generating the temporary activity label comprises using a long short-term memory (LSTM) classifier.

11. The method of claim 1 , wherein generating the predicted activity label based on the track comprises generating the predicted activity label based on state variables associated with corresponding activities.

12. The method of claim 11 , further comprising determining a location of the target based on the state variables associated with corresponding activities.

13. A millimeter-wave radar system comprising:
a millimeter-wave radar configured to transmit and receive radar signals; and
a processor comprising:
a radar processing block configured to generate a range-Doppler map based on the radar signals received by the millimeter-wave radar;
a target detector block configured to detect a target based on the range-Doppler map;
a tracker configured to:
track the target using a track, and
generate a predicted activity label based on the track, wherein the predicted activity label is indicative of an actual activity of the target;

a feature extraction block configured to generate a Doppler spectrogram based on the track;
a classifier configured to generate a temporary activity label based on the Doppler spectrogram; and
a classification gating block configured to receive an uncertainty value associated to the temporary activity label and produce gating data based on the uncertainty value, the predicted activity label, and the temporary activity label, wherein the uncertainty value is indicative of a confidence level that the temporary activity label is an actual activity of the target, and wherein the tracker is configured to generate a final activity label based on the gating data.

14. The system of claim 13 , wherein the tracker is configured to use the predicted activity label as the final activity label when the uncertainty value is above a predetermined threshold, and to use the temporary activity label as the final activity label when the uncertainty value is below the predetermined threshold.

15. The system of claim 13 , wherein the classifier comprises a first stage and a second stage, the first stage comprising a long short-term memory (LSTM) network and the second stage comprising a fully connected layer, wherein the second stage has an input coupled to an output of the first stage.

16. The system of claim 15 , wherein the first stage further comprises:
a second fully connected layer having an input coupled to the LSTM network; and
a softmax layer having an input coupled to an output of the second fully connected layer and an output coupled to the output of the first stage.

17. The system of claim 13 , wherein the radar signals comprise linear chirps.

18. A method for tracking a target using a millimeter-wave radar, the method comprising:
receiving radar signals using the millimeter-wave radar;
generating a range-Doppler map based on the received radar signals;
detecting a target based on the range-Doppler map;
using a tracker to track the target using a track;
using a classifier to generate a temporary activity label based on an output of the tracker; and
using the tracker to generate a final activity label based on an output of the classifier, wherein the tracker tracks activity labels of the target using state variables.

19. The method of claim 18 , wherein the tracker uses a predicted activity label as the final activity label when an uncertainty value associated with the temporary activity label is above a predetermined threshold, and uses the temporary activity label as the final activity label when the uncertainty value is below the predetermined threshold, wherein the tracker generates the predicted activity label based on the state variables.

20. The method of claim 18 , wherein the tracker comprises an unscented Kalman filter.

Assignees

• Infineon Technologies Ag

Inventors

• Santra Avik
• Vaishnav Prachi

Classifications

• G01S13/34 Primary

  using transmission of continuous, frequency-modulated waves while heterodyning the received signal, or a signal derived therefrom, with a locally-generated signal related to the contemporaneously transmitted signal · CPC title

• G01S7/415 Primary

  Identification of targets based on measurements of movement associated with the target · CPC title

• G06F18/2193

  based on specific statistical tests · CPC title

• G06F18/24

  Classification techniques · CPC title

• G06F18/241

  relating to the classification model, e.g. parametric or non-parametric approaches · CPC title