Robot system for processing an object and method of packaging and processing the same

Patent metadata

Field	Value
Publication number	US-10817764-B2
Application number	US-201816137673-A
Country	US
Kind code	B2
Filing date	Sep 21, 2018
Priority date	Sep 21, 2018
Publication date	Oct 27, 2020
Grant date	Oct 27, 2020

Abstract

A robot system for processing an object to be packaged as a product, a packaging method, and a method of processing the same are provided. The object has multiple surfaces, and multiple e-package information tags are provided on the surfaces of the object for storing information of the product. Each surface is provided with one of the e-package information tags. The information of the product includes information of a location, an orientation and physical features of the object. In operation, the robot system controls a sensing device to detect and capture one of the e-package information tags on the object to obtain a captured image, and processes the captured image to obtain the information of the product. Based on the information of the product, the robot system controls a robotic grasping device to perform a robotic manipulation for handling the object.

First claim

What is claimed is:

1. A packaging method, comprising:
providing an object as a product to be packaged; and
providing a plurality of e-package information tags on a plurality of surfaces of the object for storing information of the product,
wherein each of the surfaces is provided with at least one of the plurality of e-package information tags thereon, and
the information of the product comprises information of a location, an orientation and physical features of the object,
wherein each of the plurality of e-package information tags is in a form of a 2-dimensional code having a first area and a second area, the first area of each of the plurality of e-package information tags is designed to detect and estimate pose of the object; and
wherein the pose of the object is estimated by obtaining tilt angles of the plurality of e-package information tags and distance variance from a sensing device to different points on the plurality of e-package information tags and processing the tilt angles and the distance variance by the sensing device.

2. The packaging method of claim 1 , wherein the at least one of the plurality of e-package information tags provided on each of the surfaces is provided at a different relative location thereon, and with a different size, or a different shape.

3. The packaging method of claim 1 , wherein the second area of each of the plurality of e-package information tags stores the information of the physical features of the object, wherein the physical features of the object comprises:
a dimension of the object;
a weight of the object;
a weight distribution of the object;
a property of the object;
product information of the object; and
the location and the orientation of the object.

4. The packaging method of claim 1 , wherein each of the plurality of e-package information tags is in a form of a tag, a 1-dimensional barcode, or a watermark tag.

5. The packaging method of claim 1 , wherein each of the plurality of e-package information tags stores information of robotic manipulation for handling the object.

6. A robot system for processing the object provided in claim 1 , comprising:
a plurality of robotic grasping devices, each being configured to grasp the object;
a plurality of sensing devices, each being configured to obtain the information of the product; and
a controller configured to control the robotic grasping devices and the sensing devices,
wherein the controller is configured to perform:
controlling one of the sensing devices to detect and capture one of the plurality of e-package information tags on the object to obtain a captured image;
processing the captured image to obtain the tilt angles of the plurality of e-package information tags and distance variance from the one of the plurality of sensing devices to different points on the plurality of e-package information tags;
processing the tilt angles and the distance variance to determine the pose of the object; and
based on the pose of the object, controlling one of the robotic grasping devices to perform a robotic manipulation for handling the object.

7. The robot system of claim 6 , wherein each of the sensing devices is a camera, and each of the robotic grasping devices is a suction device, a robot arm, a gripper, or an electrical adhesive device.

8. A method of processing an object using a robot system, comprising:
providing the object as a product, wherein a plurality of e-package information tags is provided on a plurality of surfaces of the object for storing information of the product, each of the surfaces is provided with at least one of the plurality of e-package information tags thereon, and the information of the product comprises information of a location, an orientation and physical features of the object;
controlling a sensing device of the robot system to detect and capture one of the plurality of e-package information tags on the object to obtain a captured image;
processing the captured image to obtain tilt angles of the plurality of e-package information tags and distance variance from the sensing devices to different points on the plurality of e-package information tags;
processing the tilt angles and the distance variance to determine pose of the object; and
based on the pose of the object, controlling a robotic grasping device of the robot system to perform a robotic manipulation for handling the object.

9. The method of claim 8 , wherein the sensing device is a camera, and the robotic grasping device is a suction device, a robot arm, a gripper, or an electrical adhesive device.

10. The method of claim 8 , wherein the one of the plurality of e-package information tags provided on each of the surfaces is provided at a different relative location thereon, with a different size, or with a different shape.

11. The method of claim 8 , wherein each of the plurality of e-package information tags is in a form of a 2-dimensional code having a first area and a second area.

12. The method of claim 11 , wherein the first area of each of the plurality of e-package information tags is designed for easy detection and robust pose estimation, and the second area of each of the plurality of e-package information tags stores the information of the physical features of the object, wherein the physical features of the object comprises:
a dimension of the object;
a weight of the object;
a weight distribution of the object;
a property of the object;
product information of the object; and
the location and the orientation of the object.

13. The method of claim 8 , wherein each of the plurality of e-package information tags is in a form of a tag, a 1-dimensional barcode, or a watermark tag.

14. The method of claim 8 , wherein each of the plurality of e-package information tags stores information of the robotic manipulation for handling the object.

15. A packaging method, comprising:
providing an object as a product to be packaged; and
providing a plurality of e-package information tags on a plurality of surfaces of the object for storing information of the product, one e-package information tag being provided on each corner of each of the plurality of surfaces,
wherein the information of the product comprises information of a location, an orientation and physical features of the object;
wherein each of the plurality of e-package information tags is in a form of a 2-dimensional code having a first area and a second area, the first area of each of the plurality of e-package information tags is designed to detect and estimate pose of the object: and
wherein the pose of the object is estimated by obtaining tilt angles of the plurality of e-package information tags and distance variance from a sensing device to different points on the plurality of e-package information tags and processing the tilt angles and the distance variance by the sensing device.

16. The packaging method of claim 15 , wherein the e-package information tag comprises Apriltags or QR codes, and the e-package information tag comprises dimension, order ID, weight, weight distribution, and product information of the object.

17. The packaging method of claim 15 , wherein the e-package information tag comprises maximal speed, maximal force can be applied, fragility, property, humidity, and temperature of the object.

18. The packaging method of claim 15 , wherein each surface of the object that is operable by a robot is further provided with a surface point for determination of the surface as an operable surface and determination of a geometric center of the object.

Assignees

• Beijing Jingdong Shangke Information Technology Co Ltd
• Jd Com American Tech Corp

Inventors

• Xi Wei
• Cheng Hui

Classifications

• G06K19/06 Primary

  characterised by the kind of the digital marking, e.g. shape, nature, code · CPC title

• B25J19/023 Primary

  including video camera means · CPC title

• G06V30/224

  of printed characters having additional code marks or containing code marks · CPC title

• G06V20/194

  using hyperspectral data, i.e. more or other wavelengths than RGB · CPC title

• G06V20/10

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title