Method and apparatus for recognizing object, and method and apparatus for training recognizer

Patent metadata

Field	Value
Publication number	US-9928410-B2
Application number	US-201514938365-A
Country	US
Kind code	B2
Filing date	Nov 11, 2015
Priority date	Nov 24, 2014
Publication date	Mar 27, 2018
Grant date	Mar 27, 2018

Abstract

A recognition method includes receiving an input image; and recognizing a plurality of elements associated with the input image using a single recognizer pre-trained to recognize a plurality of elements simultaneously.

First claim

What is claimed is:

1. A recognition method comprising:
receiving an input image including a face region; and
recognizing a plurality of elements of the input image using a single recognizer including a convolutional neural network (CNN) pre-trained to recognize the plurality of elements simultaneously,
the plurality of elements including elements associated with the face region.

2. The recognition method of claim 1 , wherein the plurality of elements comprises:
an identity (ID) that identifies the input image; and
at least one attribute associated with the input image.

3. The recognition method of claim 2 , wherein the ID identifies at least an object included in the input image.

4. The recognition method of claim 2 , wherein the at least one attribute comprises at least one of:
a gender corresponding to the face region;
an age corresponding to the face region;
an ethnic group corresponding to the face region;
an attractiveness corresponding to the face region;
a facial expression corresponding to the face region; or
an emotion corresponding to the face region.

5. The recognition method of claim 4 , wherein the at least one attribute includes at least two different attributes from among the gender, the age, the ethnic group, the attractiveness, and the facial expression.

6. The recognition method of claim 1 , wherein the recognizer includes a neural network, and
the recognizing includes calculating feature values corresponding to the plurality of elements based on pre-learned weights between nodes included in the neural network.

7. The recognition method of claim 1 , wherein the recognizing includes generating a plurality of feature images based on the input image.

8. The recognition method of claim 7 , wherein the plurality of feature images comprises at least one of:
a color channel image from which illumination noise is removed;
an oriented-gradient magnitude channel image;
a skin probability channel image; or
a local binary pattern channel image.

9. The recognition method of claim 7 , wherein the recognizing comprises:
filtering the plurality of feature images; and
outputting feature values corresponding to the plurality of elements based on an output of the filtering.

10. The recognition method of claim 9 , wherein the recognizing further comprises:
recognizing the plurality of elements based on the feature values.

11. The recognition method of claim 1 , wherein the recognizing comprises:
acquiring a plurality of part images corresponding to parts of a face included in the input image; and
generating a plurality of feature images corresponding to each of the plurality of part images.

12. The recognition method of claim 11 , wherein the recognizing comprises:
outputting first feature values corresponding to the plurality of elements based on outputs of a plurality of part recognition modules,
wherein each of the plurality of part recognition modules is configured to
filter feature images of a corresponding part image; and
output feature values corresponding to elements associated with the corresponding part image based on an output of the filtering.

13. The recognition method of claim 12 , wherein the recognizing further comprises:
recognizing the plurality of elements based on the output first feature values.

14. The recognition method of claim 1 , further comprising:
comparing the plurality of elements to a plurality of elements associated with a reference image; and
determining whether the input image matches the reference image based on a result of the comparing.

15. The recognition method of claim 14 , wherein the comparing comprises:
generating a feature vector based on the plurality of elements; and
comparing the feature vector to a reference vector of the reference image.

16. A method of training a recognizer, the method comprising:
receiving a training image; and
training a recognizer configured to recognize a plurality of elements of an input image that includes a face region, based on the training image and a plurality of elements labeled in the training image, such that the recognizer includes a convolutional neural network (CNN) pre-trained to recognize the plurality of elements of the input image simultaneously,
the plurality of elements of the input image including elements associated with the face region.

17. The method of claim 16 , wherein the plurality of elements comprise:
an identity (ID) that identifies the training image; and
at least one attribute associated with the training image.

18. The method of claim 17 , wherein the ID includes information that identifies at least an object included in the training image.

19. The method of claim 17 , wherein the at least one attribute comprises at least one of:
a gender corresponding to the face region;
an age corresponding to the face region;
an ethnic group corresponding to the face region;
an attractiveness corresponding to the face region;
a facial expression corresponding to the face region; or
an emotion corresponding to the face region.

20. The method of claim 19 , wherein the at least one attribute includes at least two different attributes from among the gender, the age, the ethnic group, the attractiveness, and the facial expression.

21. The method of claim 16 , wherein the training comprises calculating losses corresponding to the plurality of elements.

22. The method of claim 21 , wherein the recognizer comprises a neural network, and
the training includes training the recognizer to learn weights between nodes included in the neural network based on the losses.

23. The method of claim 16 , wherein the recognizer comprises a neural network, and
the training includes activating nodes included in the neural network based on a stochastic piecewise linear (PWL) model.

24. The method of claim 16 , wherein the training comprises generating a plurality of feature images based on the training image.

25. The method of claim 24 , wherein the plurality of feature images comprises at least one of:
a color channel image from which illumination noise is removed;
an oriented-gradient magnitude channel image;
a skin probability channel image; or
a local binary pattern channel image.

26. The method of claim 24 , wherein the training comprises:
training the recognizer to filter the plurality of feature images; and
training the recognizer to output feature values corresponding to the plurality of elements based on an output of the filtering of the plurality of feature images.

27. The method of claim 26 , wherein the training further comprises:
training the recognizer to recognize the plurality of elements based on the output feature values.

28. The method of claim 16 , wherein the training comprises:
acquiring a plurality of part images corresponding to parts of a face included in the training image.

29. The method of claim 28 , wherein different elements are labeled in the plurality of part images.

30. The method of claim 28 , wherein the training further comprises:
generating a plurality of feature images corresponding to each of the plurality of part images.

31. The method of claim 30 , wherein the training comprises:
training the recognizer to output first feature values corresponding to the plurality of elements based on outpu

Assignees

• Samsung Electronics Co Ltd

Inventors

• Yoo Byungin
• Kim Namjoon
• Lee Changkyo
• Choi Chang Kyu
• Han Jaejoon

Classifications

• G06V10/82

  using neural networks · CPC title

• G06V40/174

  Facial expression recognition · CPC title

• G06V40/171 Primary

  Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06N3/045

  Combinations of networks · CPC title