Artificial intelligence-based process identification, extraction, and automation for robotic process automation

Patent metadata

Field	Value
Publication number	US-12337464-B2
Application number	US-202318317410-A
Country	US
Kind code	B2
Filing date	May 15, 2023
Priority date	Oct 15, 2019
Publication date	Jun 24, 2025
Grant date	Jun 24, 2025

Abstract

Artificial intelligence (AI)-based process identification, extraction, and automation for robotic process automation (RPA) is disclosed. Listeners may be deployed to user computing systems to collect data pertaining to user actions. The data collected by the listeners may then be sent to one or more servers and be stored in a database. This data may be analyzed by AI layers to recognize patterns of user behavioral processes therein. These recognized processes may then be distilled into respective RPA workflows and deployed to automate the processes.

First claim

The invention claimed is:

1. One or more computing systems, comprising:
memory storing computer program instructions; and
at least one processor configured to execute the computer program instructions, wherein the computer program instructions are configured to cause the at least one processor to:
access log data collected from a plurality of listener applications, the log data comprising information pertaining to user interactions with user computing systems,
run the log data through at least one artificial intelligence (AI) model, the at least one AI model configured to process the log data, and
identify a robotic process automation (RPA) process in the log data based on output from the at least one AI model.

2. The one or more computing systems of claim 1 , wherein the computer program instructions are further configured to cause the at least one processor to:
automatically generate an RPA workflow comprising the identified RPA process.

3. The one or more computing systems of claim 1 , wherein the at least one AI model comprises a sequence extraction model, a clustering detection model, a visual component detection model, a text recognition model, an audio-to-text translation model, or any combination thereof.

4. The one or more computing systems of claim 1 , wherein the one or more computing systems comprise a conductor application and the log data is sent periodically to the conductor application by the listener applications as part of a heartbeat message.

5. The one or more computing systems of claim 1 , wherein the one or more computing systems are configured to access the log data after a predetermined amount of log data has been collected, after a predetermined time period has elapsed, or both.

6. The one or more computing systems of claim 1 , wherein the identified RPA process comprises an atomic instance of the RPA process.

7. The one or more computing systems of claim 1 , wherein the computer program instructions are further configured to cause the at least one processor to:
determine similarities between the identified RPA process and a previously existing RPA process executed by an RPA robot on the user computing systems; and
replace an automation of the RPA robot executing the previously existing RPA process with the identified RPA process.

8. The one or more computing systems of claim 7 , wherein similarities between the identified RPA process and the previously existing RPA process are determined by at least one of a common beginning, a common end, and a predetermined amount of statistical commonality in steps taking in between in the RPA processes.

9. The one or more computing systems of claim 1 , wherein the listener applications are configured to record at least one of where a user clicks on a screen and in which application, keystrokes, which button was clicked, instances of the user switching between applications, focus changes, and that an email was sent and what the email pertains to.

10. The one or more computing systems of claim 1 , wherein the listener applications are configured to determine at least one of which applications users of the user computing systems are using, what percentage of time the users are using a given application, which features within the applications the users are using, and which features within the applications that the users are not using.

11. The one or more computing systems of claim 1 , wherein the computer program instructions are further configured to cause the at least one processor to:
list and sort identified processes by at least one of an RPA score indicating how suitable a given process is to RPA, a process name, a total recording time, a number of users who executed the process, and a process execution time.

12. A non-transitory computer-readable medium storing a computer program, the computer program configured to cause at least one processor to:
access log data collected from respective listener applications of a plurality of user computing systems;
run the log data through at least one artificial intelligence (AI) model, the at least one AI model configured to process the log data and identify one or more robotic process automation (RPA) processes therein; and
identify an RPA process in the log data based on output from the at least one AI model.

13. The non-transitory computer-readable medium of claim 12 , wherein the computer program is further configured to cause the at least one processor to:
automatically generate an RPA workflow comprising the identified RPA process.

14. The non-transitory computer-readable medium of claim 12 , wherein the at least one AI model comprises a sequence extraction model, a clustering detection model, a visual component detection model, a text recognition model, an audio-to-text translation model, or any combination thereof.

15. The non-transitory computer-readable medium of claim 12 , wherein the computer program is further configured to cause the at least one processor to:
determine similarities between the identified RPA process and a previously existing RPA process executed by an RPA robot on at least one of the user computing systems; and
replace an automation of the RPA robot executing the previously existing RPA process with the identified RPA process.

16. The non-transitory computer-readable medium of claim 12 , wherein similarities between the identified RPA process and the previously existing RPA process are determined by at least one of a common beginning, a common end, and a predetermined amount of statistical commonality in steps taking in between in the RPA processes.

17. The non-transitory computer-readable medium of claim 12 , wherein the computer program is further configured to cause the at least one processor to:
list and sort identified processes by at least one of an RPA score indicating how suitable a given process is to RPA, a process name, a total recording time, a number of users who executed the process, and a process execution time.

18. The non-transitory computer-readable medium of claim 12 , wherein the identified RPA process comprises an atomic instance of the RPA process.

19. A computer-implemented method, comprising:
accessing log data collected from respective listener applications of a plurality of user computing systems, by one or more computing systems;
running the log data through at least one artificial intelligence (AI) model, by the one or more computing systems, the at least one AI model configured to process the log data and identify one or more robotic process automation (RPA) processes therein; and
identifying an RPA process in the log data based on output from the at least one AI model, by the one or more computing systems.

20. The computer-implemented method of claim 19 , further comprising:
automatically generating an RPA workflow comprising the identified RPA process, by the one or more computing systems.

21. The computer-implemented method of claim 19 , wherein the at least one AI model comprises a sequence extraction model, a clustering detection model, a visual component detection model, a text recognition model, an audio-to-text translation model, or any combination thereof.

22. The computer implemented method of claim 19 , further comprising:
determining similarities between the identified RPA process and a previously existing RPA process executed by an RPA robot on at least one of the user computing systems, by the one or more computing systems; and
replacing an automation of the RPA robot execut

Assignees

• Uipath Inc

Inventors

• Singh Prabhdeep
• Berg Christian

Classifications

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06F16/2379

  Updates performed during online database operations; commit processing · CPC title

• G06F11/3476

  Data logging (G06F11/14, G06F11/2205 take precedence) · CPC title

• G06F11/3006 Primary

  where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems (multiprogramming arrangements G06F9/46; allocation of resources G06F9/50) · CPC title