Object tracking apparatus, object tracking method and recording medium

Patent metadata

Field	Value
Publication number	US-12243299-B2
Application number	US-201917788444-A
Country	US
Kind code	B2
Filing date	Dec 26, 2019
Priority date	Dec 26, 2019
Publication date	Mar 4, 2025
Grant date	Mar 4, 2025

Abstract

An object tracking apparatus ( 1 ) includes: a first generation unit ( 221 - 224 ) for generating, based on a first position information (PI t-τ ) related to a position of an object (O t-τ ) in a first image (IM t-τ ) captured at a first time (t-τ) and a second position information (PI t ) related to a position of an object (O t ) in a second image (IM t ) captured at a second time (t), first and second feature vectors (CV t-τ , CV t ); and a second generation unit ( 225 ) for generating, as a correspondence information (AM) indicating a correspondence relationship between the object in the first image and the object in the second image, an information that is obtained by a calculation processing using the first and the second feature vectors.

First claim

What is claimed is:

1. An object tracking apparatus comprising:
at least one memory configured to store instructions; and
at least one processor configured to execute the instructions to:
generate, based on a first position information related to a position of an object in a first image captured at a first time and a second position information related to a position of an object in a second image captured at a second time that is different from the first time, a first feature vector representing a feature of the first position information and a second feature vector representing a feature of the second position information; and
generate, as a correspondence information indicating a correspondence relationship between the object in the first image and the object in the second image, an information that is obtained by a calculation processing using the first and the second feature vectors.

2. The object tracking apparatus according to claim 1 , wherein
the calculation processing includes a processing for calculating a matrix product of the first feature vector and the second feature vector.

3. The object tracking apparatus according to claim 1 , wherein
the correspondence information indicates the correspondence relationship between the object in the first image and the object in the second image by using a matrix.

4. The object tracking apparatus according to claim 3 , wherein
the at least one processor is configured to execute the instructions to normalize the matrix by normalizing each of a vector component in one row of the matrix and a vector component in one column of the matrix by a softmax function.

5. The object tracking apparatus according to claim 1 , wherein
the correspondence information indicates the correspondence relationship between the object in the first image and the object in the second image by using a matrix in which an element reacts at a position at which a row vector component corresponding to one object in either one of the first and second images intersects with a column vector component corresponding to the one object in the other one of the first and second images,
the at least one processor is configured to execute the instructions to normalize the matrix by normalizing each of the row vector component and each the column vector component by a softmax function.

6. The object tracking apparatus according to claim 1 , wherein
the at least one processor is configured to execute the instructions to correct the second position information by using the correspondence information.

7. The object tracking apparatus according to claim 6 , wherein
the at least one processor is configured to execute the instructions to correct the second position information by using an attention mechanism that uses the correspondence information as a weight.

8. The object tracking apparatus according to claim 7 , wherein
the correspondence information indicates the correspondence relationship between the object in the first image and the object in the second image by using a matrix,
the second position information includes a position map indicating an information related to a position of the object in the second image,
the attention mechanism is configured to correct the position map that is the second position information by performing a processing for calculating a matrix product of the position map and the correspondence information.

9. The object tracking apparatus according to claim 8 , wherein
the attention mechanism is configured to correct the position map that is the second position information by performing a processing for adding a correction map, which is obtained by calculating the matrix product of the position map and the correspondence information, to the position map.

10. The object tracking apparatus according to claim 1 , wherein
the at least one processor is configured to execute the instructions to obtain, from a calculation model that outputs each of the first and second position information when the first and second images are inputted thereto, the first and second position information and an intermediate output information corresponding to an intermediate output of the calculation model,
the at least one processor is configured to execute the instructions to generate the first and second feature vectors based on the first and second position information and the intermediate output information.

11. The object tracking apparatus according to claim 1 , wherein
the at least one processor is configured to execute the instructions to:
generate the first and second position information from the first and second images;
correct the second position information by using the correspondence information; and
update a learning model that defines at least one of a first operation for generating the first and the second feature vectors, a second operation for generating the correspondence information, a third operation for generating the first and the second position information and a fourth operation for correcting the second position information based on a first loss function related to the second position information corrected by the fourth operation when learning data including the first and second images for a learning is used by the third operation.

12. The object tracking apparatus according to claim 11 , wherein
the at least one processor is configured to execute the instructions to update the learning model based on the first loss function, a second loss function related to the first position information generated by the third operation when the learning data is used by the third operation and a third loss function related to the second position information generated by the third operation when the learning data is used by the third operation,
the at least one processor is configured to execute the instructions to perform a weighting processing on the first to third loss functions so that a weight of whole of the first and third loss functions is equal to a weight of the second loss function, and to generate the learning model based on the first to third loss functions on which the weighting processing is already performed.

13. The object tracking apparatus according to claim 11 , wherein
the at least one processor is configured to execute the instructions to allow the first operation to use a plurality of learning data having different intervals between the first time and the second time to update the learning model.

14. An object tracking method including:
generating, based on a first position information related to a position of an object in a first image captured at a first time and a second position information related to a position of an object in a second image captured at a second time that is different from the first time, a first feature vector representing a feature of the first position information and a second feature vector representing a feature of the second position information; and
generating, as a correspondence information indicating a correspondence relationship between the object in the first image and the object in the second image, an information that is obtained by a calculation processing using the first and the second feature vectors.

15. A non-transitory computer readable recording medium on which a computer program that allows a computer to execute an object tracking method is recorded,
the object tracking method including:
generating, based on a first position information related to a position of an object in a first image captured at a first time and a second position information related to a position of an object in a second image

Assignees

• Nec Corp

Inventors

• Fukui Hiroshi

Classifications

• G06V2201/07

  Target detection · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T7/246

  using feature-based methods, e.g. the tracking of corners or segments · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06V10/40

  Extraction of image or video features · CPC title