Autonomous and safe integration of human task in robotic operation

Patent metadata

Field	Value
Publication number	US-12157644-B2
Application number	US-202117217474-A
Country	US
Kind code	B2
Filing date	Mar 30, 2021
Priority date	Mar 30, 2021
Publication date	Dec 3, 2024
Grant date	Dec 3, 2024

Abstract

A robotic line kitting system is disclosed. In various embodiments, a sensor reading associated with a force sensor associated with a robotic instrumentality comprising the robotic line kitting system is received. It is determined based at least in part on the sensor reading that a condition requiring human intervention has been detected. A task to be performed by a human worker to correct the condition is scheduled.

First claim

What is claimed is:

1. A robotic line kitting system, comprising:
a communication interface; and
a processor coupled to the communication interface and configured to:
receive via the communication interface a sensor reading associated with a force sensor associated with a robotic instrumentality comprising the robotic line kitting system, wherein the sensor reading associated with the force sensor occurs during a grasp of a tray that includes one or more items;
determine based at least in part on the sensor reading that a condition requiring human intervention has been detected, wherein the sensor reading indicates a weight associated with the tray, wherein a change in the weight associated with the tray is used to detect that an item of the one or more items has dropped from the tray; and
in response to the change in the weight associated with the tray indicating the condition requiring human intervention, schedule a task to be performed by a human worker to correct the condition.

2. The robotic line kitting system of claim 1 , wherein the processor is configured to determine that the condition requiring human intervention has been detected at least in part by detecting a change in a force measured by the force sensor.

3. The robotic line kitting system of claim 2 , wherein the change in the force measured by the force sensor is associated with a weight of the one or more items in a grasp of the robotic instrumentality.

4. The robotic line kitting system of claim 3 , wherein the change in force measured by the force sensor is associated with a change in detected weight of said one or more items in the grasp of the robotic instrumentality.

5. The robotic line kitting system of claim 4 , wherein the change in detected weight is from a first detected weight measured at a first time and a second non-zero detected weight computed based at least in part on the sensor reading associated with a second time after the first time.

6. The robotic line kitting system of claim 1 , wherein the condition requiring human intervention is associated with instability of a stack of receptacles and the processor is configured to detect the instability of the stack of receptacles based at least in part on the sensor reading.

7. The robotic line kitting system of claim 6 , wherein the processor is configured to detect the instability of the stack of receptacles.

8. The robotic line kitting system of claim 7 , wherein the sensor reading is associated with an operation to place one or more receptacles in a grasp of the robotic instrumentality on top of the stack of receptacles.

9. The robotic line kitting system of claim 8 , wherein the processor is configured to control the robotic instrumentality to perform a slotting operation to place the one or more receptacles in the grasp of the robotic instrumentality on top of the stack of receptacles.

10. The robotic line kitting system of claim 9 , wherein the processor is configured to detect the instability at least in part by comparing an expected reading with the received sensor reading.

11. The robotic line kitting system of claim 10 , wherein the processor is further configured to learn an association between the expected reading and successful placement of the one or more receptacles in the grasp of the robotic instrumentality on top of the stack of receptacles in a condition in which the instability is not present.

12. The robotic line kitting system of claim 1 , the task to be performed by the human worker includes a task to do one or both of checking for and retrieving the dropped item.

13. The robotic line kitting system of claim 1 , wherein the processor is further configured to mark an operating zone in which the task is to be performed as being in a state that prevents or modifies operation of the robotic instrumentality in the operating zone at a time when the task is scheduled to be performed.

14. The robotic line kitting system of claim 13 , wherein the processor is further configured to schedule the robotic instrumentality to perform, at the time when the task is scheduled to be performed, one or more other tasks in one or more operating zones other than the operating zone in which the task is to be performed.

15. The robotic line kitting system of claim 1 , wherein the processor is further configured to notify the human worker of the task scheduled to be performed by the human worker.

16. A method to control a robotic line kitting system, comprising:
receiving a sensor reading associated with a force sensor associated with a robotic instrumentality comprising the robotic line kitting system, wherein the sensor reading associated with the force sensor occurs during a grasp of a tray that includes one or more items;
determining based at least in part on the sensor reading that a condition requiring human intervention has been detected, wherein the sensor reading indicates a weight associated with the tray, wherein a change in the weight associated with the tray is used to detect that an item of the one or more items has dropped from the tray; and
in response to the change in the weight associated with the tray indicating the condition requiring human intervention, scheduling a task to be performed by a human worker to correct the condition.

17. The method of claim 16 , wherein the condition requiring human intervention has been detected at least in part by detecting a change in a force measured by the force sensor.

18. The method of claim 17 , wherein in the change in force measured by the force sensor is associated with a change in detected weight of said one or more items in the grasp of the robotic instrumentality.

19. The method of claim 16 , wherein the condition requiring human intervention is associated with instability of a stack of receptacles and the instability is detected based at least in part on the sensor reading.

20. A computer program product to control a robotic line kitting system, the computer program product being embodied in a non-transitory computer readable medium and comprising computer instructions for:
receiving a sensor reading associated with a force sensor associated with a robotic instrumentality comprising the robotic line kitting system, wherein the sensor reading associated with the force sensor occurs during a grasp of a tray that includes one or more items;
determining based at least in part on the sensor reading that a condition requiring human intervention has been detected, wherein the sensor reading indicates a weight associated with the tray, wherein a change in the weight associated with the tray is used to detect that an item of the one or more items has dropped from the tray; and
in response to the change in the weight associated with the tray indicating the condition requiring human intervention, scheduling a task to be performed by a human worker to correct the condition.

Assignees

• Dexterity Inc

Inventors

• Sun Zhouwen
• Sun Cuthbert
• Nguyen Andrew
• Tay Samuel
• Peck Emilie
• Chavez Kevin Jose
• Holmberg Robert

Classifications

• B25J9/1674

  characterised by safety, monitoring, diagnostic · CPC title

• B65G2203/0258

  Weight of the article · CPC title

• B25J9/1679

  characterised by the tasks executed · CPC title

• B25J9/1697

  Vision controlled systems · CPC title

• B65G2203/041

  Camera · CPC title