Method and apparatus for recognizing object

Patent metadata

Field	Value
Publication number	US-10691971-B2
Application number	US-201715492772-A
Country	US
Kind code	B2
Filing date	Apr 20, 2017
Priority date	Nov 28, 2016
Publication date	Jun 23, 2020
Grant date	Jun 23, 2020

Abstract

A method includes actuating a processor to apply an input image to a feature extractor including a plurality of layers, determine a third feature vector based on first feature vectors of an input image output by a first layer included in a feature extractor and second feature vectors of the input image output by a second layer in the feature extractor, and identify an object in the input image based on the third feature vector.

First claim

What is claimed is:

1. A processor-implemented method of recognizing an object, the method comprising:
applying an input image to a feature extractor including a plurality of layers;
extracting first feature vectors from the input image by one or more first layers of the feature extractor;
extracting second feature vectors from pooling results with respect to the extracted first feature vector by one or more second layers of the feature extractor;
determining a third feature vector based on the extracted first feature vectors and the extracted second feature vectors; and
identifying the object in the input image based on the third feature vector,
wherein the determining of the third feature vector comprises:
determining first sub-vectors and second sub-vectors based on corresponding values having a substantially identical offset in the extracted first feature vectors;
obtaining a feature vector associated with the first layer based on operations using the first sub-vectors and second the sub-vectors; and
determining the third feature vector based on the feature vector associated with the first layer.

2. The method of claim 1 , wherein the determining of the third feature vector comprises determining the third feature vector by performing a pooling operation with respect to each of the first feature vectors and the second feature vectors.

3. The method of claim 1 , wherein the determining of the third feature vector further comprises:
obtaining a first pooled vector by performing a pooling operation with respect to each of the first feature vectors corresponding to results of a convolutional operation based on kernels by a convolutional layer of the one or more first layers;
obtaining a second pooled vector by performing a pooling operation with respect to each of the second feature vectors corresponding to results of a convolutional operation based on other kernels by a convolutional layer of the one or more second layers; and
determining the third feature vector based on the first pooled vector and the second pooled vector.

4. The method of claim 3 , wherein the determining of the third feature vector further comprises lightening the first pooled vector and the second pooled vector.

5. The method of claim 4 , wherein the first pooled vector and the second pooled vector are lightened based on a principal component analysis (PCA).

6. The method of claim 3 , wherein the obtaining of the first pooled vector comprises obtaining the first pooled vector by performing the pooling operation with respect to each of the first feature vectors based on the first feature vectors and a mask vector for passing an object corresponding to a target category.

7. A processor-implemented method of recognizing an object, the method comprising:
applying an input image to an object separator;
obtaining a mask image indicating a category of at least one object included in the input image;
obtaining a mask vector based on the mask image;
applying the input image to a feature extractor including a plurality of layers; and
determining a third feature vector based on a first pooled feature vector of first feature vectors of the input image output by a first layer included in the feature extractor and a second pooled feature vector of second feature vectors of the input image output by a second layer included in the feature extractor,
wherein the first pooled feature vector is a result of pooling based on the first feature vectors and the mask vector.

8. The method of claim 3 , wherein the obtaining of the first pooled vector comprises obtaining the first pooled vector by performing the pooling operation with respect to each of the first feature vectors based on the first feature vectors and mask vectors for passing each target part of the object.

9. A processor implemented method of recognizing an object, the method comprising:
applying an input image to a feature extractor including a plurality of layers;
determining a third feature vector based on first feature vectors of the input image output by a first layer included in the feature extractor and second feature vectors of the input image output by a second layer included in the feature extractor; and
identifying the object in the input image based on the third feature vector,
wherein the determining of the third feature vector comprises:
determining column vectors and row vectors based on corresponding points having a substantially identical offset in the first feature vectors;
obtaining a plurality of vectors based on respective products of the column vectors and the row vectors;
determining a feature vector associated with the first layer based on the obtained plurality of vectors; and
determining the third feature vector based on the feature vector associated with the first layer.

10. The method of claim 1 , wherein the recognizing of the object comprises:
applying the third feature vector to a nonlinear classifier including a plurality of layers; and
recognizing the object in the input image based on an output of the nonlinear classifier.

11. The method of claim 10 , wherein the output of the nonlinear classifier indicates a specific category corresponding to the object in the input image among specific categories included in a predetermined category.

12. A non-transitory computer-readable storage medium storing instructions, that, when executed by the processor, cause the processor to perform the method of claim 1 .

13. An object recognizing apparatus comprising:
a processor; and
a memory comprising an instruction executable by the processor, wherein, in response to the instruction being executed by the processor, the processor is configured to:
apply an input image to a feature extractor comprising a plurality of layers;
extract first feature vectors from the input image by one or more first layers of the feature extractor;
extract second feature vectors from pooling results with respect to the extracted first feature vector by one or more second layers of the feature extractor;
determine a third feature vector based on the extracted first feature vectors and the extracted second feature vectors; and
identify the object in the input image based on the third feature vector,

wherein, in response to the instruction being executed by the processor, the processor is further configured to:
determine first sub-vectors and second sub-vectors based on corresponding values having a substantially identical offset in the extracted first feature vectors;
obtain a feature vector associated with the first layer based on operations using the first sub-vectors and second the sub-vectors; and
determine the third feature vector based on the feature vector associated with the first layer.

14. The object recognizing apparatus of claim 13 , wherein, in response to the instruction being executed by the processor, the processor is further configured to determine the third feature vector by performing a pooling operation with respect to each of the first feature vectors and the second feature vectors.

15. The object recognizing apparatus of claim 13 , wherein, in response to the instruction being executed by the processor, the processor is further configured:
to obtain a first pooled vector by performing a pooling operation with respect to each of the first feature vectors,
to obtain a second pooled vector by performing the pooling operation with respect to each of the second feature vectors, and
to determine the third feature vector based on the first pooled vector and the second pooled vector.

16. The object recognizing apparat

Assignees

• Samsung Electronics Co Ltd

Inventors

• Chang Hyun Sung
• Sagong Donghoon
• Son Minjung
• Jung Kyungboo
• Sung Young Hun

Classifications

• G06V10/82

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06V10/25 Primary

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06F18/21355

  nonlinear criteria, e.g. embedding a manifold in a Euclidean space · CPC title