Target detection method, apparatus, and system

Patent metadata

Field	Value
Publication number	US-11367272-B2
Application number	US-202016854815-A
Country	US
Kind code	B2
Filing date	Apr 21, 2020
Priority date	Jan 30, 2018
Publication date	Jun 21, 2022
Grant date	Jun 21, 2022

Abstract

A target detection method and apparatus, in which the method includes: obtaining a target candidate region in a to-be-detected image; determining at least two part candidate regions from the target candidate region by using an image segmentation network, where each part candidate region corresponds to one part of a to-be-detected target; and extracting, from the to-be-detected image, local image features corresponding to the part candidate regions; and learning the local image features of the part candidate regions by using a bidirectional long short-term memory LSTM network, to obtain a part relationship feature used to describe a relationship between the part candidate regions; and detecting the to-be-detected target in the to-be-detected image based on the part relationship feature. As a result, image data processing precision in target detection can be improved, application scenarios of target detection can be diversified, and target detection accuracy can be improved.

First claim

What is claimed is:

1. A method, comprising:
obtaining a target candidate region in a to-be-detected image;
determining at least two part candidate regions from the target candidate region by using an image segmentation network, wherein each part candidate region corresponds to one part of a to-be-detected target; and extracting, from the to-be-detected image, local image features corresponding to the part candidate regions;
learning the local image features of the part candidate regions by using a bidirectional long short-term memory (LSTM) network, to obtain a part relationship feature used to describe a relationship between the part candidate regions; and
detecting the to-be-detected target in the to-be-detected image based on the part relationship feature.

2. The method according to claim 1 , wherein the detecting the to-be-detected target in the to-be-detected image based on the part relationship feature comprises:
determining the to-be-detected target in the to-be-detected image based on the part relationship feature with reference to a global image feature, wherein the global image feature corresponds to the target candidate region; and
the detecting the to-be-detected target in the to-be-detected image based on the part relationship feature further comprises:
obtaining the global image feature corresponding to the target candidate region.

3. The method according to claim 2 , wherein the detecting the to-be-detected target in the to-be-detected image based on the part relationship feature with reference to a global image feature comprises:
merging the part relationship feature with the global image feature, and obtaining, through learning, a first confidence level of each of a category and a location of the to-be-detected target in the to-be-detected image based on a merged feature;
determining, based on the global image feature, a second confidence level at which the target candidate region comprises the to-be-detected target; and determining, based on merging of the first confidence level and the second confidence level, that the to-be-detected image comprises the to-be-detected target; and
determining a location of the to-be-detected target in the to-be-detected image based on a location of the target candidate region in the to-be-detected image.

4. The method according to claim 1 , wherein the learning the local image features of the part candidate regions by using the LSTM network comprises:
sorting the local image features of the part candidate regions in a preset sequence to obtain a sorted feature sequence, and inputting the feature sequence to the LSTM network; and
learning, by using the LSTM network, the relationship between the part candidate regions by using a binary classification problem distinguishing between a target and a background as a learning task.

5. The method according to claim 4 , wherein the relationship between the part candidate regions comprises at least one of a relationship between the to-be-detected target and the part candidate regions, or a dependency relationship between the part candidate regions.

6. A method, comprising:
obtaining a target candidate region in a to-be-detected image;
obtaining a positive sample image feature and a negative sample image feature that are used for part identification, and constructing a part identification model based on the positive sample image feature and the negative sample image feature;
identifying at least two part candidate regions from the target candidate region by using the part identification model, wherein each part candidate region corresponds to one part of a to-be-detected target; and extracting, from the to-be-detected image, local image features corresponding to the part candidate regions;
learning the local image features of the part candidate regions by using a bidirectional long short-term memory (LSTM) network, to obtain a part relationship feature used to describe a relationship between the part candidate regions; and
detecting the to-be-detected target in the to-be-detected image based on the part relationship feature.

7. The method according to claim 6 , wherein the obtaining the positive sample image feature and the negative sample image feature that are used for part identification comprises:
obtaining a candidate box template, dividing the candidate box template into N grids, and determining, from the N grids, a grid covered by a region in which each part of the target is located, wherein N is an integer greater than 1;
obtaining a sample image used for part identification, and determining a plurality of candidate regions from the sample image;
determining a candidate region labeled with the target in the plurality of candidate regions as a positive sample region of the target, and determining a candidate region whose intersection-over-union with the positive sample region is less than a preset proportion as a negative sample region of the target;
dividing the positive sample region into N grids, and determining, from the N grids of the positive sample region based on the candidate box template, a positive sample grid and a negative sample grid that correspond to each part;
dividing the negative sample region into N grids, and determining a grid that is in the N grids of the negative sample region and that corresponds to a respective part as a negative sample grid of the part; and
determining an image feature of a positive sample grid region of each part as a positive sample image feature of the part, and determining an image feature of a negative sample grid region of each part as a negative sample image feature of the part.

8. The method according to claim 7 , wherein the determining, from the N grids of the positive sample region based on the candidate box template, the positive sample grid and the negative sample grid that correspond to each part comprises:
determining, from the N grids of the positive sample region based on a grid that is in the candidate box template and that is covered by a region in which each part is located, one or more part grids covered by the part; and
when one or more part grids covered by any part i comprises a part grid j, and a degree at which a region covered by the part i in the part grid j overlaps a region of the part grid j is greater than or equal to a preset threshold, determining the part grid j as a positive sample grid of the part i, to determine a positive sample grid of each part, wherein both i and j are natural numbers.

9. The method according to claim 7 , wherein the determining, from the N grids of the positive sample region based on the candidate box template, the positive sample grid and the negative sample grid that correspond to each part comprises:
determining, from the N grids of the positive sample region based on a grid that is in the candidate box template and that is covered by a region in which each part is located, one or more part grids covered by the part; and
when one or more part grids covered by any part i comprises a part grid j, and a degree at which a region covered by the part i in the part grid j overlaps a region of the part grid j is less than a preset threshold, determining the part grid j as a negative sample grid of the part i, to determine a negative sample grid of each part, wherein both i and j are natural numbers.

10. The method according to claim 6 , wherein the constructing the part identification model based on the positive sample image feature and the negative sample image feature comprises:
using the positive sample image feature of each part and the negative sample image feature of each part as input of the part identification model, and learning, by using the part identification model and by using a bi

Assignees

• Huawei Tech Co Ltd

Inventors

• Yang Yi
• Jiang Yuhao
• Chen Maolin
• Yang Shuang

Classifications

• G06V10/40

  Extraction of image or video features · CPC title

• G06V10/776

  Validation; Performance evaluation · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06V40/10 Primary

  Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands · CPC title

• G06V10/50 Primary

  by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis · CPC title