Apparatus and method for identifying and picking object using artificial intelligence algorithm

Patent metadata

Field	Value
Publication number	US-11645778-B2
Application number	US-201816767421-A
Country	US
Kind code	B2
Filing date	Nov 6, 2018
Priority date	Dec 21, 2017
Publication date	May 9, 2023
Grant date	May 9, 2023

Abstract

According to various embodiments of the present invention, an electronic device comprises: a memory including instructions and a training database, which includes data, on at least one object, acquired on the basis of an artificial intelligence algorithm; at least one sensor; and a processor connected to the at least one sensor and the memory, wherein the processor can be configured to execute the instructions in order to acquire data on a designated area including the at least one object by using the at least one sensor, identify location information and positioning information on the at least one object on the basis of the training database, and transmit a control signal for picking the at least one object to a picking tool related to the electronic device on the basis of the identified location information and positioning information.

First claim

What is claimed is:

1. An electronic device comprising:
at least one sensor;
a memory configured to store instructions and a training database, the training database comprising training data on an object obtained by processing simulated sensor data for an arrangement of multiple ones of the object based on a first artificial intelligence algorithm; and
a processor coupled with the at least one sensor and the memory,
wherein the processor is configured to execute the instructions to cause the electronic device to:
obtain virtual data representing a form and a shape of the object;
obtain sensor data by performing a physical simulation to the virtual data, wherein the physical simulation includes at least one operation for acquiring realistic information on the object by using physical numeric values;
perform a video simulation to the sensor data obtained by the physical simulation and real data related to the object, based on the first artificial intelligence algorithm;
obtain the training data by performing the video simulation;
acquire data on a specified area including a plurality of instances of the object, using the at least one sensor;
identify location information and positioning information on at least one object of the plurality of instances of the object included in the specified area, based on a second artificial intelligence algorithm using the training data in the training database;
provide a location map, based on the identified location information, wherein the location map comprises at least one point representing a location in which the at least one object is put within the specified area;
provide a position map, based on the identified positioning information, wherein the position map comprises at least one arrow representing a direction in which the at least one object is put; and
transmit a control signal for picking the at least one object to a picking tool, based on the location map and the position map.

2. The electronic device of claim 1 , wherein the at least one sensor comprises a sensor configured to sense at least one of whether the object is included in the specified area, a shape of the object, a location of the object, or a position of the object.

3. The electronic device of claim 1 , wherein the data on the specified area comprises image information of an image related to the specified area.

4. The electronic device of claim 1 , wherein the location information of the at least one object comprises at least one of an x-axis value, a y-axis value, or a z-axis value, on the specified area, of the at least one object, and
the positioning information of the at least one object comprises at least one of information on a yaw of the at least one object, information on a roll of the at least one object, or information on a pitch of the at least one object.

5. The electronic device of claim 1 , wherein the instructions further cause the electronic device to:
provide the location map and the position map, based on the second artificial intelligence algorithm.

6. The electronic device of claim 1 , wherein the instructions further cause the electronic device to:
process the data on the specified area, the location information of the at least one object, and the positioning information of the at least one object, based on the second artificial intelligence algorithm; and
update the training database based on the processing, and
wherein the training database comprises the location information of the at least one object, the positioning information of the at least one object, and video information of the at least one object, and
the video information comprises virtual data representing the at least one object, and information provided based on the second artificial intelligence algorithm.

7. A method of an electronic device, the method comprising:
obtaining a training database, the training database comprising training data on an object obtained by processing simulated sensor data for an arrangement of multiple ones of the object based on a first artificial intelligence algorithm;
obtaining virtual data representing a form and a shape of the object;
obtaining sensor data by performing a physical simulation to the virtual data, wherein the physical simulation includes at least one operation for acquiring realistic information on the object by using physical numeric values;
performing a video simulation to the sensor data obtained by the physical simulation and real data related to the object, based on the first artificial intelligence algorithm, wherein the training data is obtained by performing the video simulation;
acquiring data on a specified area including a plurality of instances of the object, using at least one sensor;
identifying location information and positioning information on at least one object of the plurality of instances of the object included in the specified area, based on a second artificial intelligence algorithm using the training data in the training database;
providing a location map, based on the identified location information, wherein the location map comprises at least one point representing a location in which the at least one object is put within the specified area;
providing a position map, based on the identified positioning information, wherein the position map comprises at least one arrow representing a direction in which the at least one object is put; and
transmitting a control signal for picking the at least one object to a picking tool, based on the location map and the position map.

8. The method of claim 7 , wherein the at least one sensor comprises a sensor configured to sense at least one of whether the object is included in the specified area, a shape of the object, a location of the object, or a position of the object.

9. The method of claim 7 , wherein the data on the specified area comprises image information of an image related to the specified area.

10. The method of claim 7 , wherein the location information of the at least one object comprises at least one of an x-axis value, a y-axis value, or a z-axis value on the specified area of the at least one object, and
the positioning information of the at least one object comprises at least one of information on a yaw of the at least one object, information on a roll of the at least one object, or information on a pitch of the at least one object.

11. The method of claim 10 , further comprising:
processing the data on the specified area, the location information of the at least one object, and the positioning information of the at least one object, based on the second artificial intelligence algorithm; and
updating the training database based on the processing,
wherein the training database comprises the location information of the at least one object, the positioning information of the at least one object, and video information of the at least one object, and
the video information comprises virtual data representing the at least one object, and information provided based on the second artificial intelligence algorithm.

12. The method of claim 7 , further comprising:
providing the location map and the position map, based on the second artificial intelligence algorithm.

13. A non-transitory computer-readable storage medium for storing one or more programs which, when executed by a processor of an electronic device, cause the electronic device to:
obtain a training database, the training database comprising training data on an object obtained by processing simulated sensor data for an arrangement of multiple ones of the object based on a first artificial intelligence algorithm;
obtain virtual data represe

Assignees

• Samsung Electronics Co Ltd

Inventors

• Min Jun Hong
• Park Soon Yong
• Kim Ji Min
• Roh Kyung Shik
• Bahn Wook

Classifications

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0475

  Generative networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/094

  Adversarial learning · CPC title