Methods and systems for object tracking

Patent metadata

Field	Value
Publication number	US-12499689-B2
Application number	US-202318183014-A
Country	US
Kind code	B2
Filing date	Mar 13, 2023
Priority date	Mar 30, 2022
Publication date	Dec 16, 2025
Grant date	Dec 16, 2025

Abstract

The present disclosure relates to methods and systems for object tracking, for example for object detection and grid segmentation using recurrent neural networks. A computer implemented method for object tracking comprises the following steps carried out by computer hardware components: providing random values as a hidden state of a trained neural network for an initial time step, wherein the hidden state represents an encoding of sensor data acquired over consecutive time steps in a grid structure, wherein the hidden state further represents an offset indicating a movement of the object between the consecutive time steps; iteratively determining an updated hidden state by processing a present hidden state and present sensor data using the trained neural network; and determining object tracking information based on the updated hidden state.

First claim

What is claimed is:

1 . A computer implemented method for tracking an object, the method comprising:
providing random values as a hidden state of a trained neural network for an initial time step, the hidden state representing: an encoding of sensor data acquired over consecutive time steps in a grid structure, and an offset indicating a movement of the object between the consecutive time steps; iteratively determining an updated hidden state by processing a present hidden state and present sensor data using the trained neural network; determining object tracking information based on the updated hidden state; acquiring the present hidden state for a first pre-determined time step and an offset for the first pre-determined time step; determining a transformed hidden state based on the present hidden state for the first pre-determined time step and the offset for the first pre-determined time step; acquiring sensor data for a second pre-determined time step; evaluating the trained neural network based on the transformed hidden state and the sensor data for the second pre-determined time step to obtain intermediate data for the second pre-determined time step; determining an offset for the second pre-determined time step based on the intermediate data; and determining the updated hidden state for the second pre-determined time step based on the intermediate data and the offset for the second pre-determined time step.

2 . The method of claim 1 , wherein the offset for the first pre-determined time step is determined based on the present hidden state for the first pre-determined time step.

3 . The method of claim 1 , wherein the updated hidden state is determined further based on at least one of:
concatenating the intermediate data and the offset for the second pre-determined time step; the offset for the first pre-determined time step; a mean of the offset for the first pre-determined time step and the offset for the second pre-determined time step; or an attention method based on the offset for the first pre-determined time step and the offset for the second pre-determined time step.

4 . The method of claim 1 , further comprising:
determining second intermediate data based on the transformed hidden state using a matching method; and evaluating the trained neural network based on the second intermediate data.

5 . The method of claim 4 , wherein using the matching method further comprises:
determining a similarity between an embedding vector of the first pre-determined time step and an embedding vector of the second pre-determined time step.

6 . The method of claim 5 , wherein the embedding vector of the first pre-determined time step comprises an embedding based on the transformed hidden state for the first pre-determined time step.

7 . The method of claim 5 , wherein the embedding vector of the second pre-determined time step comprises an embedding based on the sensor data for the second pre-determined time step.

8 . The method of claim 5 , wherein the similarity is determined based on a dot product between the embedding vector of the first pre-determined time step and the embedding vector of the second pre-determined time step.

9 . The method of claim 1 , wherein the updated hidden state for the second pre-determined time step is determined based on speed matching.

10 . The method of claim 1 , further comprising:
performing object detection based on the updated hidden state for the second pre-determined time step.

11 . The method of claim 1 , further comprising:
performing grid segmentation based on the updated hidden state for the second pre-determined time step.

12 . The method of claim 1 , wherein the offset for the second
pre-determined time step is determined based on sampling.

13 . The method of claim 12 , wherein the sampling is based on a Gaussian distribution.

14 . The method of claim 1 , wherein the trained neural network comprises a recurrent neural network.

15 . The method of claim 14 , wherein the recurrent neural network comprises at least one of long short-term memory (LSTM) or gated recurrent units (GRUs).

16 . The method of claim 1 , wherein the object comprises an ego vehicle.

17 . The method of claim 1 , wherein the object comprises an object with movements.

18 . A computer system comprising:
a plurality of computer hardware components including a processor; and a non-transitory computer readable medium comprising instructions, which when executed by the processor, cause the processor to: provide random values as a hidden state of a trained neural network for an initial time step, the hidden state representing:
an encoding of sensor data acquired over consecutive time steps in a grid structure, and
an offset indicating a movement of an object between the consecutive time steps;
iteratively determine an updated hidden state by processing a present hidden state and present sensor data using the trained neural network; determine object tracking information for the object based on the updated hidden state; acquire the present hidden state for a first pre-determined time step and an offset for the first pre-determined time step; determine a transformed hidden state based on the present hidden state for the first pre-determined time step and the offset for the first pre-determined time step; acquire sensor data for a second pre-determined time step; evaluate the trained neural network based on the transformed hidden state and the sensor data for the second pre-determined time step to obtain intermediate data for the second pre-determined time step; determine an offset for the second pre-determined time step based on the intermediate data; and determine the updated hidden state for the second pre-determined time step based on the intermediate data and the offset for the second pre-determined time step.

19 . A vehicle comprising:
a sensor; a processor; a non-transitory computer-readable medium comprising instructions, which when executed by the processor cause the processor to:
provide random values as a hidden state of a trained neural network for an initial time step, the hidden state representing:
an encoding of sensor data from the sensor acquired over consecutive time steps in a grid structure, and
an offset indicating a movement of an object between the consecutive time steps;

iteratively determine an updated hidden state by processing a present hidden state and present sensor data using the trained neural network; determine object tracking information for the object based on the updated hidden state; acquire the present hidden state for a first pre-determined time step and an offset for the first pre-determined time step; determine a transformed hidden state based on the present hidden state for the first pre-determined time step and the offset for the first pre-determined time step; acquire sensor data for a second pre-determined time step; evaluate the trained neural network based on the transformed hidden state and the sensor data for the second pre-determined time step to obtain intermediate data for the second pre-determined time step; determine an offset for the second pre-determined time step based on the intermediate data; and determine the updated hidden state for the second pre-determined time step based on the intermediate data and the offset for the second pre-determined time step.

Assignees

• Aptiv Technologies AG

Inventors

• Braun Marco
• Luszek Moritz
• Spata Dominic

Classifications

• G06V2201/07

  Target detection · CPC title

• G06V10/82

  using neural networks · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/09

  Supervised learning · CPC title