Semantic-based method and apparatus for retrieving perspective image

Patent metadata

Field	Value
Publication number	US-12131543-B2
Application number	US-202117202088-A
Country	US
Kind code	B2
Filing date	Mar 15, 2021
Priority date	Mar 16, 2020
Publication date	Oct 29, 2024
Grant date	Oct 29, 2024

Abstract

A semantic-based method and apparatus for retrieving a perspective image, an electronic device and a computer-readable storage medium are provided. An method includes obtaining a perspective image for a space containing an inspected object therein. A semantic division on the perspective image is performed using a first method, to obtain a plurality of semantic region units. A feature extraction network is constructed using a second method. Based on the perspective image and each of the plurality of semantic region units, a feature of each semantic region unit is extracted using the feature extraction network. Based on the feature of each semantic region unit, an image most similar to the semantic region unit is retrieved from an image feature database, to assist in determining an inspected object in the semantic region unit.

First claim

What is claimed is:

1. A semantic-based method for retrieving a perspective image, comprising: in a cargo inspection process,
obtaining, by a perspective image acquisition apparatus, a perspective image for a space containing an inspected object therein;
performing a semantic division on the perspective image using a first method, to obtain a plurality of semantic region units;
constructing a feature extraction network using a second method;
extracting, based on the perspective image and each of the plurality of semantic region units, a feature of each semantic region unit using the feature extraction network;
fixing a feature dimension of the feature of each semantic region unit, to normalize the feature of each semantic region unit to the same feature dimension; and
retrieving, based on the feature of each semantic region unit, an image most similar to the semantic region unit from an image feature database, to assist in determining an inspected object in the semantic region unit, wherein each feature in the image feature database uses each divided semantic region unit as a basic unit, and each feature in the image feature database has information describing a cargo category, a region image attribution, declaration information of an image where the semantic region unit is located, and coordinates of an object region in the image,
wherein the extracting, based on the perspective image and each of the plurality of semantic region units, a feature of each semantic region unit using the feature extraction network comprises:
obtaining, based on the perspective image, a feature map for the perspective image using the feature extraction network; and
obtaining, based on coordinates of each semantic region unit in the perspective image, the feature of the semantic region unit in the feature map.

2. The method of claim 1 , wherein the first method is one or more of:
selective search, objectness method, and region proposal net RPN.

3. The method of claim 1 , wherein the second method is an introduction of a feature pyramid network FPN into a basic network Resnet.

4. The method of claim 1 , wherein the image feature database is established based on history perspective images without suspicious objects, and
wherein the perspective images used for establishing the image feature database have one or more item information recorded.

5. The method of claim 4 , wherein the image most similar to the semantic region unit is retrieved from the image feature database based on the item information.

6. The method of claim 1 , wherein there are a predetermined number of images most similar to the semantic region unit existed.

7. The method of claim 1 , further comprising:
displaying, by a display unit, information related to the inspected object.

8. An electronic device, comprising:
one or more processors; and
a memory configured to store one or more programs;
wherein when the one or more programs are executed by the one or more processors, cause the one or more processors to implement the method of claim 1 .

9. A non-transitory computer-readable storage medium having executable instructions stored thereon which, when executed by a processor, cause the processor to implement the method of claim 1 .

Assignees

• Univ Tsinghua
• Nuctech Co Ltd

Inventors

• Zhang Li
• Chen Zhiqiang
• Li Yuanjing
• Xing Yuxiang
• Meng Fanhua
• Li Qiang
• Li Wei
• Fu Gang

Classifications

• G06V10/255

  Detecting or recognising potential candidate objects based on visual cues, e.g. shapes · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06V10/75

  Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries · CPC title

• G06V20/20

  in augmented reality scenes · CPC title