Apparatus and method with image processing and target tracking

Patent metadata

Field	Value
Publication number	US-12567158-B2
Application number	US-202218077465-A
Country	US
Kind code	B2
Filing date	Dec 8, 2022
Priority date	Dec 10, 2021
Publication date	Mar 3, 2026
Grant date	Mar 3, 2026

Abstract

An apparatus and method with tracking a target are provided. A method includes determining whether data augmentation is to be used to augment a target tracking process, based on determining that data augmentation is to be used, performing the target tracking process based on an augmented image area obtained by the data augmentation on an image area, and outputting a tracking result generated by the target tracking process.

First claim

What is claimed is:

1 . A target tracking method, the method comprising:
determining whether data augmentation is to be used to augment a target tracking process; based on determining that data augmentation is to be used, performing the target tracking process based on an augmented image area obtained by the data augmentation on an image area, wherein the performing of the target tracking process based on the augmented image area comprises:
obtaining N tracking results by performing the target tracking process based on N augmented image areas:
determining a first augmented tracking result by comparing the N tracking results to one another; and
outputting the first augmented tracking result as a tracking result responsive to a first augmentation confidence score included in the first augmented tracking result being within a second preset score interval; and
outputting the tracking result generated by the target tracking process.

2 . The method of claim 1 , further comprising, outputting, as the tracking result, a first original tracking result obtained from the target tracking process based on the image area, when it is determined that the data augmentation is not to be used.

3 . The method of claim 1 , wherein the determining the data augmentation is to be used comprises:
obtaining a first original tracking result by performing the target tracking process based on the image area; and determining whether the data augmentation is to be used according to the first original tracking result.

4 . The method of claim 3 , wherein the first original tracking result comprises a first original predicted position of a tracked target and a first original confidence score corresponding to the first original predicted position of the tracked target, and
wherein the determining of whether the data augmentation is to be used is based on the first original confidence score.

5 . The method of claim 4 , wherein the determining of whether the data augmentation is to be used, according to the first original confidence score, comprises:
when the first original confidence score is within a first preset score range, determining that the data augmentation is to be used.

6 . The method of claim 1 , wherein
the N augmented image areas are obtained by augmenting data on the image area by using N data augmentation processing methods, respectively.

7 . The method of claim 6 , wherein the outputting of the tracking result further comprises:
outputting a first original tracking result as the tracking result responsive to the first augmented confidence score included in the first augmented tracking result being not within the second preset score interval.

8 . The method of claim 7 , wherein the image area comprises a template image area comprising a tracked target within a frame image or a search area within the frame image.

9 . The method of claim 8 , further comprising, when the image area is a first search area positioned in a t-th frame image and the first augmented tracking result is output as the tracking result:
obtaining a second original tracking result corresponding to a second search area of a t+1th frame image by performing the target tracking process based on the second search area; augmenting data on the second search area through a data augmentation processing method corresponding to the first augmented tracking result; determining a second augmented tracking result corresponding to the second search area by performing the target tracking based on the augmented second search area; and determining a final tracking result corresponding to the second search area, from the second original tracking result and the second augmented tracking result.

10 . The method of claim 9 , further comprising:
when the second augmented tracking result is determined to be the final tracking result corresponding to the second search area, augmenting data on a third search area in a t+2th frame image through the data augmentation processing method corresponding to the first augmented tracking result and determining a third augmented tracking result corresponding to the third search area to be a final tracking result corresponding to the third search area by performing the target tracking process based on the augmented third search area; and when the second original tracking result is determined to be the final tracking result corresponding to the second search area, determining a final tracking result corresponding to the third image search area by performing the target tracking based on an original third search area.

11 . The method of claim 8 , further comprising, when the image area is a template image area and the first augmented tracking result corresponding to the first search area in a t-th frame image is output as the tracking result:
obtaining a second original tracking result corresponding to a second search area of a t+1th frame image by performing the target tracking based on the second search area and the template image area; determining a second augmented tracking result corresponding to the second search area by performing the target tracking based on the second search area and an augmented template image area; and determining a final tracking result corresponding to the second search area, from the second original tracking result and the second augmented tracking result, wherein the augmented template image area is obtained by augmenting data on the template image area through a data augmentation processing method corresponding to the first augmented tracking result.

12 . The method of claim 11 , further comprising:
when the second original tracking result is determined to be the final tracking result corresponding to the second search area, determining a final tracking result corresponding to a third search area in a t+2th frame image by performing the target tracking process based on the third search area and the template image area; and when the second augmented tracking result is determined to be the final tracking result corresponding to the second search area, determining the final tracking result corresponding to the third search area by performing the target tracking process based on the third search area and the augmented template image area.

13 . The method of claim 6 , wherein the N augmented image areas are obtained by augmenting data on the image area, using the N data augmentation processing methods, respectively, through obtaining an augmented image area corresponding to the image area by augmenting the image area, using augmentation chains for each of the N data augmentation processing methods, respectively.

14 . The method of claim 13 , wherein the obtaining of the augmented image area corresponding to the image area by augmenting the image area using the augmentation chains comprises:
performing an augmentation processing on the image area using each augmentation chain; and obtaining the augmented image area corresponding to the image area by performing weighted combination output results of the respective augmentation chains.

15 . The method of claim 14 , wherein at least one of the augmentation chains is randomly selected from among augmentation chain candidates.

16 . The method of claim 15 , wherein each of the augmentation chain candidates is linked to at least one augmentation primitive among a plurality of augmentation primitives.

17 . The method of claim 16 , wherein the augmentation primitives comprise a contrast primitive, a color primitive, a brightness primitive, a sharpness primitive, a clipping primitive,

Assignees

• Samsung Electronics Co Ltd

Inventors

• Yu Jiaqian
• Chen Yiwei
• Park Changbeom
• Lee Hyunjeong
• Yoo Byung In
• Wang Qiang
• Pan Siyang

Classifications

• G06T2207/30221

  Sports video; Sports image · CPC title

• G06T2207/30196

  Human being; Person · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T7/248 Primary

  involving reference images or patches · CPC title