System and method for picking and placing objects

Patent metadata

Field	Value
Publication number	US-12583109-B2
Application number	US-202218091143-A
Country	US
Kind code	B2
Filing date	Dec 29, 2022
Priority date	Jun 21, 2022
Publication date	Mar 24, 2026
Grant date	Mar 24, 2026

Abstract

A system and a method for picking and placing objects, includes capturing at least one first image of the object at a first time point and at least one second image at a second time point; identifying the object in the first image and the second image, obtaining first position information, second position information, and object attribute of the object; based on the first position information, the second position information, the first time point, and the second time point, calculating a moving speed of the object; based on the object attribute, determining whether the object is a target object; if so, calculating a target position and a target time point of the target object based on the first and second position information and the moving speed; controlling a target machine arm to move at or to the target position at the target time point to pick the target object.

First claim

What is claimed is:

1 . A method for picking and placing objects applied in a system for picking and placing objects, the system comprising a plurality of machine arms and a transmission device, the transmission device being configured to transmit at least one object in a predetermined direction, the method comprising:
capturing, by an imaging device arranged above the transmission device, at least one first image of the at least one object at a first time point; capturing, by the imaging device, at least one second image of the at least one object at a second time point; identifying, by an operating and processing module connected to the transmission device, the imaging device, and the plurality of machine arms, the at least one object in the at least one first image and the at least one second image; and obtaining first position information, second position information, and at least one object attribute of the at least one object; based on the first position information, the second position information, the first time point, and the second time point, calculating, by the operating and processing module, a moving speed of the at least one object; based on the at least one object attribute of the at least one object, determining, by the operating and processing module, whether the at least one object is a target object; in a case that the at least one object is determined to be the target object, calculating, by the operating and processing module, a target position and a target time point of the target object based on the first position information, the second position information, and the moving speed; selecting, by the operating and processing module, a target machine arm from the plurality of machine arms; and controlling, by the operating and processing module, the target machine arm selected from the plurality of machine arms to move at or to the target position at the target time point to pick the target object.

2 . The method according to claim 1 , wherein the at least one object attribute of the at least one object comprises material, color, class, and shape.

3 . The method according to claim 1 , further comprising:
controlling, by the operating and processing module, the target machine arm selected from the plurality of machine arms to place the target object to a corresponding storage bunker.

4 . The method according to claim 1 , wherein the first position information, the second position information, and the at least one object attribute of the at least one object are obtained by:
obtaining, by the operating and processing module, depth information of the at least one object in the at least one first image and the at least one second image, and the moving speed of the at least one object is calculated based on a time difference between the at least one first image and the at least one second image.

5 . The method according to claim 1 , wherein each first image of the at least one first image and each second image of the at least one second image is a two-dimensional and three-dimensional fused image comprising depth information.

6 . The method according to claim 1 , wherein selecting the target machine arm from the plurality of machine arms comprises:
relating, by the operating and processing module, the plurality of machine arms with at least one picking attribute; and based on a related picking attribute of the at least one machine arm and the at least one object attribute of the target object, selecting, by the operating and processing module, the target machine arm from the plurality of machine arms.

7 . The method according to claim 6 , wherein each machine arm of the plurality of machine arms has a picking attribute of the at least one picking attribute different from the picking attributes of the at least one picking attribute of other machine arms of the plurality of machine arms.

8 . The method according to claim 1 , wherein selecting the target machine arm from the plurality of machine arms comprises:
obtaining, by the operating and processing module, picking-and-placing areas of each machine arm of the plurality of machine arms; based on the picking-and-placing areas of the plurality of machine arms and the target position of the target object, selecting, by the operating and processing module, the target machine arm from the plurality of machine arms.

9 . The method according to claim 8 , wherein the picking-and-placing areas of each machine arm of the plurality of machine arms do not overlap with each other the picking-and-placing areas of other machine arms of the plurality of machine arms.

10 . The method according to claim 8 , wherein the picking-and-placing areas of each machine arm of the plurality of machine arms partially overlaps with the picking-and-placing areas of other machine arms of the plurality of machine arms.

11 . A system for picking and placing objects, the system comprising:
a transmission device configured to transmit at least one object in a predetermined direction; an imaging device arranged above the transmission device, the imaging device configured to capture at least one first image of the at least one object at a first time point and at least one second image of the at least one object at a second time point; a plurality of machine arms, each of the plurality of machine arms configured to pick and place the at least one object; and an operating and processing module connected to the transmission device, the imaging device, and the plurality of machine arms, the operating and processing module configured to: identify the at least one object in the at least one first image and the at least one second image; and obtaining first position information, second position information, and at least one object attribute of the at least one object; based on the first position information, the second position information, the first time point, and the second time point, calculate a moving speed of the at least one object; based on the at least one object attribute of the at least one object, determine whether the at least one object is a target object; in a case that the at least one object is determined to be the target object, calculate a target position and a target time point of the target object based on the first position information, the second position information, and the moving speed; select a target machine arm from the plurality of machine arms; and control the target machine arm selected from the plurality of machine arms to move at or to the target position at the target time point to pick the target object.

12 . The system according to claim 11 , wherein the at least one object attribute of the at least one object comprises material, color, class, and shape.

13 . The system according to claim 11 , wherein the operating and processing module is further configured to:
control the target machine arm selected from the plurality of machine arms to place the target object to a corresponding storage bunker.

14 . The system according to claim 11 , wherein the operating and processing module is further configured to:
obtain depth information of the at least one object in the at least one first image and the at least one second image, and based on a time difference between the at least one first image and the at least one second image, calculate the moving speed of the at least one object.

15 . The system according to claim 11 , wherein each first image of the at least one first image and each second image of the at least one second image is a two-dimensional and three-dimensional fused image comprising depth i

Assignees

• Chiun Mai Communication Systems Inc

Inventors

• Hsieh Tung-Chun
• Wu Chung-Wei
• Yu Hsin-Ko
• Li Chih-Wei
• Lee Sung-Chuan
• Lo Tze-Chin
• Ko Chien-Ming

Classifications

• B25J1/08

  movably mounted in a wall · CPC title

• B25J9/16

  Program controls (total factory control, i.e. centrally controlling a plurality of machines, G05B19/418) · CPC title

• G06T7/593

  from stereo images · CPC title

• G06T7/55

  from multiple images · CPC title

• G06T7/90

  Determination of colour characteristics · CPC title