Visual search method, computer device, and storage medium

Patent metadata

Field	Value
Publication number	US-11348254-B2
Application number	US-201917041411-A
Country	US
Kind code	B2
Filing date	Jul 1, 2019
Priority date	Nov 21, 2018
Publication date	May 31, 2022
Grant date	May 31, 2022

Abstract

A visual search method, a computer device, and a non-transitory computer readable storage medium are provided. An ith image frame is received. The location and the classification of the subject in the ith image frame are extracted. A detection block corresponding to the subject is generated. In subsequent image frames of the ith image frame, the subject is tracked on the basis of the location of the subject in the ith image frame. The detection block is adjusted on the basis of the tracking result.

First claim

What is claimed is:

1. A method for video searching, comprising:
receiving an i th image frame, i being a positive integer;
extracting a location and a classification of a subject in the i th image frame, and generating a detection block corresponding to the subject;
obtaining an (i+n) th image frame after the i th image frame, where n is a positive integer;
tracking the subject according to the location of the subject in the (i+n) th image frame;
obtaining image frames between an (i+1) th image frame and an (i+n−1) th image frame as reference image frames;
verifying the tracking of the subject according to the reference image frames, comprising:
for each pair of two adjacent reference image frames, determining a variation range of the location of the subject between the two adjacent reference image frames; comparing the variation ranges corresponding to adjacent pairs to determine a difference; and determining whether the difference is within an allowable error range;

and
adjusting the detection block according to a tracking result.

2. The method according to claim 1 , further comprising:
receiving an (i+M) th image frame, M being a positive integer;
determining whether a subject in the (i+M) th image frame changes relative to the subject in the i th image frame; and
in response to changing, regenerating a detection block according to the subject detected in the (i+M) th image frame, and re-tracking the subject in the (i+M) th image frame.

3. The method according to claim 1 , wherein in the subsequent image frames of the i th image frame, tracking the subject according to the location of the subject in the i th image frame comprises:
obtaining brightness of each subsequent image frame;
in response to determining that a difference between the brightness of two adjacent image frames is greater than or equal to a first preset threshold, calling a kernelized correlation filters tracking algorithm to track the subject according to the location of the subject in the i th image frame; and
in response to determining that a difference between the brightness of two adjacent image frames is less than the first preset threshold, calling an optical flow tracking algorithm to track the subject according to the location of the subject in the i th image frame.

4. The method according to claim 1 , further comprising:
determining whether the location of the subject in one subsequent image frame is different from the location of the subject in the i th image frame;
in response to determining that the location of the subject in the subsequent image frame is different from the location of the subject in the i th image frame, adjusting the detection block according to the location of the subject in the subsequent image frame.

5. The method according to claim 1 , wherein a plurality of subjects are comprised and a plurality of detection blocks are generated.

6. The method according to claim 5 , wherein the plurality of subjects have respective unique identification identifiers.

7. A computer device, comprising a processor and a memory, wherein the processor runs programs corresponding to executable program codes by reading the executable program codes stored in the memory, to
receive an i th image frame, i being a positive integer;
extract a location and a classification of a subject in the i th image frame, and generate a detection block corresponding to the subject;
obtain an (i+n) th image frame after the i th image frame, where n is a positive integer;
track the subject according to the location of the subject in the (i+n) th image frame;
obtain image frames between an (i+1) th image frame and an (i+n−1) th image frame as reference image frames;
verify the tracking of the subject according to the reference image frames, by:
for each pair of two adjacent reference image frames, determining a variation range of the location of the subject between the two adjacent reference image frames; comparing the variation ranges corresponding to adjacent pairs to determine a difference; and determining whether the difference is within an allowable error range; and

adjust the detection block according to a tracking result.

8. The computer device according to claim 7 , wherein the processor is further configured to:
receive an (i+M) th image frame, M being a positive integer;
determine whether a subject in the (i+M) th image frame changes relative to the subject in the i th image frame; and
in response to changing, regenerate a detection block according to the subject detected in the (i+M) th image frame, and re-track the subject in the (i+M) th image frame.

9. The computer device according to claim 7 , wherein the processor is further configured to:
obtain brightness of each subsequent image frame;
in response to determining that a difference between the brightness of two adjacent image frames is greater than or equal to a first preset threshold, call a kernelized correlation filters tracking algorithm to track the subject according to the location of the subject in the i th image frame; and
in response to determining that a difference between the brightness of two adjacent image frames is less than the first preset threshold, call an optical flow tracking algorithm to track the subject according to the location of the subject in the i th image frame.

10. The computer device according to claim 7 , wherein the processor is further configured to:
determine whether the location of the subject in one subsequent image frame is different from the location of the subject in the i th image frame;
in response to determining that the location of the subject in the subsequent image frame is different from the location of the subject in the i th image frame, adjust the detection block according to the location of the subject in the subsequent image frame.

11. The computer device according to claim 7 , wherein a plurality of subjects are comprised and a plurality of detection blocks are generated.

12. The computer device according to claim 11 , wherein the plurality of subjects have respective unique identification identifiers.

13. A non-transitory computer-readable storage medium, having computer programs stored thereon, wherein when the programs are executed by a processor, the visual search method is implemented, the visual search method comprising:
receiving an i th image frame, i being a positive integer;
extracting a location and a classification of a subject in the i th image frame, and generating a detection block corresponding to the subject;
obtaining an (i+n) th image frame after the i th image frame, where n is a positive integer;
tracking the subject according to the location of the subject in the (i+n) th image frame;
obtaining image frames between an (i+1) th image frame and an (i+n−1) th image frame as reference image frames;
verifying the tracking of the subject according to the reference image frames, comprising:
for each pair of two adjacent reference image frames, determining a variation range of the location of the subject between the two adjacent reference image frames; comparing the variation ranges corresponding to adjacent pairs to determine a difference; and determining whether the difference is within an allowable error range; and

adjusting the detection block according to a tracking result.

14. The non-transitory computer-readable storage medium according to claim 13 , wherein the method further comprises:
receiving an (i+M) th image frame, M being a positive integer;
determining whether a subject in the (i+M) th image frame changes relative to the

Assignees

• Baidu online network technology beijing co ltd

Inventors

• Zhang Liuqing
• Li Guohong
• Qiu Xin
• Gao Shuhui
• Zhang Yazhou

Classifications

• G06T7/238 Primary

  using non-full search, e.g. three-step search · CPC title

• G06T7/223 Primary

  using block-matching · CPC title

• G06V20/40

  in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44) · CPC title

• G06V10/56

  relating to colour · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title