Human-in-the-loop robot training for robotic process automation

Patent metadata

Field	Value
Publication number	US-11815880-B2
Application number	US-202117506292-A
Country	US
Kind code	B2
Filing date	Oct 20, 2021
Priority date	Oct 15, 2019
Publication date	Nov 14, 2023
Grant date	Nov 14, 2023

Abstract

Human-in-the-loop robot training using artificial intelligence (AI) for robotic process automation (RPA) is disclosed. This may be accomplished by a listener robot watching interactions of a user or another robot with a computing system. Based on the interactions by the user or robot with the computing system, the robot may be improved and/or personalized for the user or a group of users.

First claim

The invention claimed is:

1. A cloud robotic process automation (RPA) system, comprising:
a user computing system comprising an RPA robot and a listener; and
one or more cloud computing systems configured to perform human-in-the-loop RPA robot training using artificial intelligence (AI), wherein
the listener is configured to:
monitor user interactions with the RPA robot via the user computing system and log data pertaining to the interactions, and
transmit the logged data pertaining to the user interactions to the one or more cloud computing systems, and

the one or more cloud computing systems are configured to:
receive the logged data pertaining to the user interactions,
determine whether a modification should be made to an RPA workflow for the RPA robot based on a predetermined number of exceptions of a same type being received by one or more users in the logged data, based on an exception frequency, or both, and
when the one or more cloud computing systems determine that the modification should be made and the modification is addressable by inserting an activity or sequence of activities into the RPA workflow for the RPA robot, insert the activity or sequence of activities into the RPA workflow for the RPA robot that makes the determined modification.

2. The cloud RPA system of claim 1 , wherein the one or more cloud computing systems are further configured to:
generate a new version of the RPA robot using the modified RPA workflow; and
deploy the new version of the RPA robot to the user computing system.

3. The cloud RPA system of claim 1 , wherein the user computing system is configured to:
receive a new version of the RPA robot from the one or more cloud computing systems; and
deploy the new version of the RPA robot.

4. The cloud RPA system of claim 1 , wherein the logged data comprises exceptions noted by the user via the user computing system during operation of the RPA robot.

5. The cloud RPA system of claim 4 , wherein the exceptions pertain to errors by the RPA robot, user preferences, or both.

6. The cloud RPA system of claim 1 , wherein when the modification is not addressable by inserting the activity or sequence of activities into the RPA workflow, the one or more cloud computing systems are further configured to:
train a local machine learning (ML) model based on the logged data; and
modify the RPA workflow to call the trained ML model.

7. The cloud RPA system of claim 1 , wherein the one or more cloud computing systems are further configured to:
collect logged data pertaining to interactions of other users of other computing systems with respective RPA robots,
when exceptions for the user are similar to those in the collected logged data for a group of the other users that is a subset of all of the other users:
train a community ML model for the subset of users, and
modify the RPA workflow to call the community model, and

when exceptions for the user are similar to those in the collected logged data for a group of the other users and exceeds a global retraining threshold:
train a global ML model for all users, and
modify the RPA workflow to call the global model.

8. The cloud RPA system of claim 1 , wherein the logged data is transmitted to the one or more cloud computing systems by the listener as part of a heartbeat message to a conductor application running on one or more cloud computing systems.

9. A non-transitory computer-readable medium storing a computer program, the computer program configured to cause at least one processor to:
monitor user interactions with an RPA robot via a user computing system and log data pertaining to the interactions, the logged data comprising exceptions;
transmit the logged data pertaining to the user interactions to one or more cloud computing systems of a cloud RPA system;
receive a new version of the RPA robot from the one or more cloud computing systems of the cloud RPA system that has been modified to address the exceptions in the logged data responsive to receiving a predetermined number of exceptions of a same type, an exception frequency, or both; and
deploy the new version of the RPA robot.

10. The non-transitory computer-readable medium of claim 9 , wherein the exceptions pertain to errors by the RPA robot, user preferences, or both.

11. The non-transitory computer-readable medium of claim 9 , wherein the logged data is transmitted to the one or more cloud computing systems of the cloud RPA system as part of a heartbeat message to a conductor application running on the one or more cloud computing systems.

12. The non-transitory computer-readable medium of claim 9 , wherein the logged data is sent to the one or more cloud computing systems of the cloud RPA system after a predetermined amount of data has been collected, after a predetermined time period has elapsed, or both.

13. A computer-implemented method for performing human-in-the-loop robotic process automation (RPA) robot training using artificial intelligence (AI), comprising:
receiving, by one or more cloud computing systems of a cloud RPA system, logged data pertaining to interactions of a user with an RPA robot;
determining, by the one or more cloud computing systems, whether a modification should be made to an RPA workflow for the RPA robot based on a predetermined number of exceptions of a same type being received by one or more users in the logged data, based on an exception frequency, or both; and
when the one or more cloud computing systems determine that the modification should be made and the modification is addressable by inserting an activity or sequence of activities into the RPA workflow for the RPA robot, inserting the activity or sequence of activities into the RPA workflow for the RPA robot that makes the determined modification, by the one or more cloud computing systems.

14. The computer-implemented method of claim 13 , further comprising:
generating a new version of the RPA robot, by the one or more cloud computing systems, using the modified RPA workflow; and
deploying the new version of the RPA robot, by the one or more cloud computing systems.

15. The computer-implemented method of claim 13 , wherein the logged data comprises exceptions noted by the user during operation of the RPA robot.

16. The computer-implemented method of claim 15 , wherein the exceptions pertain to errors by the RPA robot, user preferences, or both.

17. The computer-implemented method of claim 13 , wherein when the modification is not addressable by inserting the activity or sequence of activities into the RPA workflow, the method further comprises:
training a local machine learning (ML) model based on the logged data, by the one or more cloud computing systems; and
modifying the RPA workflow to call the trained ML model, by the one or more cloud computing systems.

18. The computer-implemented method of claim 13 , further comprising:
collecting logged data pertaining to interactions of other users with respective RPA robots, by the one or more cloud computing systems;
when exceptions for the user are similar to those in the collected logged data for a group of the other users that is a subset of all of the other users:
training a community ML model for the subset of users, by the one or more cloud computing systems, and
modifying the RPA workflow to call the community model, by the one or more cloud computing systems; and

when exceptions for the user are similar to those in the collected logged data for a group of the other users and exceeds a global retraining threshold:
training a g

Assignees

• Uipath Inc

Inventors

• Singh Prabhdeep
• Kunnath Liji
• Kadakia Palak

Classifications

• G05B19/4155 Primary

  characterised by program execution, i.e. part program or machine function execution, e.g. selection of a program · CPC title

• B25J9/163

  learning, adaptive, model based, rule based expert control · CPC title

• G06F8/60

  Software deployment · CPC title

• G06N20/00

  Machine learning · CPC title

• G05B2219/39371

  Host and robot controller · CPC title