Method and apparatus for recognizing object

Patent metadata

Field	Value
Publication number	US-11216694-B2
Application number	US-201816633825-A
Country	US
Kind code	B2
Filing date	Jul 11, 2018
Priority date	Aug 8, 2017
Publication date	Jan 4, 2022
Grant date	Jan 4, 2022

Abstract

The present disclosure relates to an artificial intelligence (AI) system for simulating functions of a human brain such as cognition and decision-making by using machine learning algorithms such as deep learning, and applications thereof. In particular, the present disclosure provides a method of recognizing an object by using an AI system and its application, including: extracting pieces of first feature information respectively regarding a plurality of images, each image including an object; generating at least one piece of second feature information representing a correlation between the plurality of images by combining together the extracted pieces of first feature information respectively regarding the plurality of images; and recognizing, based on the at least one piece of second feature information, the object included in each of the plurality of images by using a pre-generated learning network model.

First claim

What is claimed is:

1. An object recognition method comprising:
extracting, by using a pre-generated first learning network model, a plurality of pieces of first feature information respectively regarding a target image and a plurality of images;
generating at least one piece of second feature information representing a correlation between the target image and one of the plurality of images; and
recognizing, by applying the at least one piece of second feature information as input data, an object included in the target image by using a pre-generated second learning network model,
wherein each of the target image and the plurality of images includes the object and each of the plurality of images is associated with the target image; and
wherein the piece of second feature information is generated by combining the piece of first feature information regarding the target image and the piece of first feature information regarding the one of the plurality of images.

2. The object recognition method of claim 1 , wherein the generating of the at least one piece of second feature information comprises:
generating the at least one piece of second feature information representing at least one of a similarity and a difference between the target image and the one of the plurality of images,
wherein the at least one piece of second feature information is generated by linearly combining the piece of first feature information regarding the target image and the piece of first feature information respectively regarding the one of the plurality of images.

3. The object recognition method of claim 1 , wherein the generating of the at least one piece of second feature information comprises:
comparing with one another the extracted pieces of first feature information respectively regarding the plurality of images; and
selecting, from among the plurality of images, a plurality of images in which a difference between their corresponding pieces of first feature information is within a preset range; and
combining the piece of first feature information regarding the target image and the pieces of first feature information respectively regarding the selected plurality of images.

4. The object recognition method of claim 1 , further comprising determining a region where the object is located in each of the plurality of images,
wherein the extracting of each piece of first feature information comprises extracting the first feature information of the object from the determined region.

5. The object recognition method of claim 1 , wherein the recognizing of the object comprises:
setting a weight for the at least one piece of second feature information being input to the pre-generated second learning network model according to a type of the combination of the pieces of first feature information based on which the at least one piece of second feature information is generated; and
recognizing the object by applying the at least one piece of second feature information to the pre-generated second learning network model according to the set weight.

6. The object recognition method of claim 1 , wherein the recognizing of the object comprises recognizing the object by using the pre-generated second learning network model, by applying the at least one piece of second feature information and pieces of first feature information regarding at least some of the plurality of images as input data.

7. The object recognition method of claim 1 , further comprising:
acquiring a plurality of captured training images of different shapes of the object according to at least one of a characteristic of the object, a movement of the object, and a movement of a photographing device; and
determining, based on the acquired plurality of training images, parameters for a plurality of layers comprising the second learning network model.

8. An object recognition apparatus comprising:
a memory storing one or more instructions;
an output unit; and
a processor configured to execute the one or more instructions stored in the memory to:
extract, by using a pre-generated first learning network model, a plurality of pieces of first feature information respectively regarding a target image and a plurality of images;
generate at least one piece of second feature information representing a correlation between the target image and one of the plurality of images; and
recognize, by applying the at least one piece of second feature information as input data, an object included in the target image by using a pre-generated second learning network model,
wherein each of the target image and the plurality of images includes the object and each of the plurality of images is associated with the target image; and
wherein the piece of second feature information is generated by combining the piece of first feature information regarding the target image and the piece of first feature information regarding the one of the plurality of images.

9. The object recognition apparatus of claim 8 , wherein the processor is further configured to execute the one or more instructions to:
generate the at least one piece of second feature information representing at least one of a similarity and a difference between the target image and the one of the plurality of images,
wherein the at least one piece of second feature information is generated by linearly combining the piece of first feature information regarding the target image and the piece of first feature information respectively regarding the one of the plurality of images.

10. The object recognition apparatus of claim 8 , wherein the processor is further configured to execute the one or more instructions to:
compare with one another the extracted pieces of first feature information respectively regarding the plurality of images and selecting, from among the plurality of images, a plurality of images in which a difference between their corresponding pieces of first feature information is within a preset range; and
combine the piece of first feature information regarding the target image and the pieces of first feature information respectively regarding the selected plurality of images.

11. The object recognition apparatus of claim 8 , wherein the processor is further configured to execute the one or more instructions to:
determine a region where the object is located in each of the plurality of images; and
extract the first feature information of the object from the determined region.

12. The object recognition apparatus of claim 8 , wherein the processor is further configured to execute the one or more instructions to:
set a weight for the at least one piece of second feature information being input to the pre-generated second learning network model according to a type of the combination of the pieces of first feature information based on which the at least one piece of second feature information is generated; and
recognize the object by applying the at least one piece of second feature information to the pre-generated second learning network model according to the set weight.

13. The object recognition apparatus of claim 8 , wherein the processor is further configured to execute the one or more instructions to recognize the object by using the pre-generated second learning network model, by applying the at least one piece of second feature information and pieces of first feature information regarding at least some of the plurality of images as input data.

14. The object recognition apparatus of claim 8 , wherein the processor is further configured to execute the one or more instructions to:
acquire a plurality of captured training images of

Assignees

• Samsung Electronics Co Ltd

Inventors

• Jung Hyun-Joo
• Lee Gun-Hee
• Choi In-Kwon
• Kim Sung-Jin
• Choi Hyun-Soo

Classifications

• G06V20/30 Primary

  in albums, collections or shared content, e.g. social network photos or video · CPC title

• G06V10/10

  Image acquisition (document image scanning and transmission H04N1/00; control of digital cameras H04N23/60) · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title