Reinforcement learning based conveyoring control

Patent metadata

Field	Value
Publication number	US-11685605-B2
Application number	US-202016874401-A
Country	US
Kind code	B2
Filing date	May 14, 2020
Priority date	May 14, 2020
Publication date	Jun 27, 2023
Grant date	Jun 27, 2023

Abstract

Various embodiments described herein relate to techniques for reinforcement learning based conveyoring control. In this regard, a conveyor system is configured to transport one or more objects via a conveyor belt. Furthermore, a vision system comprises one or more sensors configured to scan the one or more objects associated with the conveyor system. A processing device is configured to employ a machine learning model to determine object pose data associated with the one or more objects. The processing device is further configured to generate speed control data for the conveyor belt of the conveyor system based on a set of control policies associated with the object pose data.

First claim

What is claimed is:

1. A system, comprising:
a conveyor system configured to transport one or more objects via a conveyor belt;
a vision system that comprises one or more sensors configured to scan the one or more objects associated with the conveyor system; and
a processing device configured to employ a machine learning model to determine object pose data associated with the one or more objects based on sensor data captured by the one or more sensors of the vision system, wherein the object pose data comprises a position and/or an orientation of each of the one or more objects with respect to the conveyor belt, and wherein the processing device is further configured to generate speed control data for the conveyor belt of the conveyor system based on a set of control policies associated with the object pose data, wherein the speed control data comprises a belt speed for the conveyor belt.

2. The system of claim 1 , wherein the processing device is configured to employ a convolutional neural network to determine object pose data associated with the one or more objects.

3. The system of claim 1 , wherein the processing device is configured to employ the machine learning model to determine position data associated with the one or more objects based on the one or more images associated with the one or more objects.

4. The system of claim 1 , wherein the processing device is configured to employ the machine learning model to determine orientation data associated with the one or more objects based on the one or more images associated with the one or more objects.

5. The system of claim 1 , wherein the processing device is configured to determine object pose data associated with the one or more objects based on RGB sensor data generated by the vision system.

6. The system of claim 1 , wherein the machine learning model is a first machine learning model, and wherein the processing device is configured to generate the set of control policies based on a second machine learning model associated with reinforcement learning related to a plurality of conveyor systems.

7. The system of claim 6 , wherein the second machine learning model is trained based on simulated data associated with the plurality of conveyor systems.

8. The system of claim 1 , wherein the one or more objects is one or more first objects, wherein the conveyor system is a first conveyor system, and wherein the vision system scans the one or more first objects provided via the first conveyor system and one or more second objects provided via a second conveyor system.

9. The system of claim 1 , wherein the processing device is configured to provide a control signal associated with the speed control data to an actuator of the conveyor system.

10. A computer-implemented method, comprising:
receiving, by a device comprising a processor, sensor data associated with one or more objects transported via a conveyor belt of a conveyor system;
determining, by the device, object pose data associated with the one or more objects by employing a machine learning model that infers the object pose data based on the sensor data, wherein the object pose data comprises a position and/or an orientation of each of the one or more objects with respect to the conveyor belt; and
generating, by the device, speed control data for the conveyor belt of the conveyor system based on a set of control policies associated with the object pose data, wherein the speed control data comprises a belt speed for the conveyor belt.

11. The computer-implemented method of claim 10 , wherein the determining the object pose data comprises employing a convolutional neural network that infers the object pose data based on the sensor data.

12. The computer-implemented method of claim 10 , wherein the determining the object pose data comprises determining position data associated with the one or more objects based on the sensor data.

13. The computer-implemented method of claim 10 , wherein the determining the object pose data comprises determining orientation data associated with the one or more objects based on the sensor data.

14. The computer-implemented method of claim 10 , further comprising:
receiving, by the device, the sensor data from a vision system that scans the conveyor system.

15. The computer-implemented method of claim 10 , wherein the machine learning model is a first machine learning model, and wherein the computer-implemented method further comprises:
generating, by the device, the set of control policies based on a second machine learning model associated with reinforcement learning related to a plurality of conveyor systems.

16. The computer-implemented method of claim 11 , further comprising:
training, by the device, the second machine learning model based on simulated data associated with the plurality of conveyor systems.

17. The computer-implemented method of claim 10 , further comprising:
providing, by the device, a control signal associated with the speed control data to an actuator of the conveyor system.

18. A computer program product comprising at least one computer-readable storage medium having program instructions embodied thereon, the program instructions executable by a processor to cause the processor to:
receive sensor data associated with one or more objects transported via a conveyor belt of a conveyor system;
determine object pose data associated with the one or more objects by employing a machine learning model that infers the object pose data based on the sensor data, wherein the object pose data comprises a position and/or an orientation of each of the one or more objects with respect to the conveyor belt; and
generate speed control data for the conveyor belt of the conveyor system based on a set of control policies associated with the object pose data, wherein the speed control data comprises a belt speed for the conveyor belt.

19. The computer program product of claim 18 , wherein the program instructions are executable by the processor to cause the processor to:
determine position data associated with the one or more objects based on the sensor data.

20. The computer program product of claim 18 , wherein the program instructions are executable by the processor to cause the processor to:
determine orientation data associated with the one or more objects based on the sensor data.

Assignees

• Intelligrated Headquarters Llc

Inventors

• Makhal Abhijit
• Swain Dikshya
• Lakhmani Sagar

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/092

  Reinforcement learning · CPC title

• G06N3/006

  based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO] · CPC title

• B65G2203/041

  Camera · CPC title