Systems and methods for robotic process automation of mobile platforms

Patent metadata

Field	Value
Publication number	US-11977904-B2
Application number	US-202318312869-A
Country	US
Kind code	B2
Filing date	May 5, 2023
Priority date	May 28, 2020
Publication date	May 7, 2024
Grant date	May 7, 2024

Abstract

In some embodiments, a robotic process automation (RPA) design application provides a user-friendly graphical user interface that unifies the design of automation activities performed on desktop computers with the design of automation activities performed on mobile computing devices such as smartphones and wearable computers. Some embodiments connect to a model device acting as a substitute for an actual automation target device (e.g., smartphone of specific make and model) and display a model GUI mirroring the output of the respective model device. Some embodiments further enable the user to design an automation workflow by directly interacting with the model GUI.

First claim

The invention claimed is:

1. A method comprising employing at least one hardware processor of a computer system to:
display a desktop graphical user interface (GUI) and a model GUI on a screen of the computer system, wherein the model GUI mimics a display of a mobile computing device targeted for robotic process automation (RPA);
construct an RPA workflow comprising a first RPA activity for interacting with a desktop computer and a second RPA activity for interacting with the mobile computing device, wherein constructing the RPA workflow comprises:
receiving a first user input to the desktop GUI,
in response to the first user input, configuring the first RPA activity according to the first user input,
receiving a second user input to the model GUI, and
in response to the second user input, configuring the second RPA activity according to the second user input; and

generate at least one robot specification according to the RPA workflow, the at least one robot specification instructing a first RPA robot to carry out the first RPA activity on the desktop computer and instructing a second RPA robot to carry out the second RPA activity on the mobile computing device.

2. The method of claim 1 , wherein the RPA workflow performs a two-factor authentication procedure on the desktop computer and mobile computing device.

3. The method of claim 1 , further comprising, in preparation for displaying the model GUI, employing the at least one hardware processor to:
establish a connection with an RPA model device sharing a set of hardware and software characteristics with the mobile computing device; and
retrieve a content of the model GUI from the RPA model device.

4. The method of claim 3 , wherein the RPA model device comprises a software emulation of the mobile computing device.

5. The method of claim 1 , further comprising employing the at least one hardware processor to display a graphical representation of the RPA workflow on the screen of the computer system.

6. The method of claim 5 , wherein the graphical representation of the RPA workflow comprises a first container grouping RPA activities of the RPA workflow targeting the desktop computer and second container grouping RPA activities of the RPA workflow targeting the mobile computing device.

7. The method of claim 1 , further comprising employing the at least one hardware processor to:
display a first activity menu listing RPA activities for interacting with the desktop computer, and a second activity menu listing RPA activities for interacting with the mobile computing device;
configure the first RPA activity further according to a user's selection from the first activity menu; and
configure the second RPA activity further according to a user's selection from the second activity menu.

8. The method of claim 1 , further comprising employing the at least one hardware processor to, in response to the second user input, update the model GUI to show a result of the second user input.

9. The method of claim 1 , wherein the second RPA activity comprises executing a user gesture for interacting with the mobile computing device.

10. A computer system comprising at least one hardware processor configured to:
display a desktop graphical user interface (GUI) and a model GUI on a screen of the computer system, wherein the model GUI mimics a display of a mobile computing device targeted for robotic process automation (RPA);
construct an RPA workflow comprising a first RPA activity for interacting with a desktop computer and a second RPA activity for interacting with the mobile computing device, wherein constructing the RPA workflow comprises:
receiving a first user input to the desktop GUI,
in response to the first user input, configuring the first RPA activity according to the first user input,
receiving a second user input to the model GUI, and
in response to the second user input, configuring the second RPA activity according to the second user input; and

generate at least one robot specification according to the RPA workflow, the at least one robot specification instructing a first RPA robot to carry out the first RPA activity on the desktop computer and instructing a second RPA robot to carry out the second RPA activity on the mobile computing device.

11. The computer system of claim 10 , wherein the RPA workflow performs a two-factor authentication procedure on the desktop computer and mobile computing device.

12. The computer system of claim 10 , wherein the at least one hardware processor is further configured, in preparation for displaying the model GUI, to:
establish a connection with an RPA model device sharing a set of hardware and software characteristics with the mobile computing device; and
retrieve a content of the model GUI from the RPA model device.

13. The computer system of claim 12 , wherein the RPA model device comprises a software emulation of the mobile computing device.

14. The computer system of claim 10 , wherein the at least one hardware processor is further configured to display a graphical representation of the RPA workflow on the screen of the computer system.

15. The computer system of claim 14 , wherein the graphical representation of the RPA workflow comprises a first container grouping RPA activities of the RPA workflow targeting the desktop computer and a second container grouping RPA activities of the RPA workflow targeting the mobile computing device.

16. The computer system of claim 10 , wherein the at least one hardware processor is further configured to:
display a first activity menu listing RPA activities for interacting with the desktop computer, and a second activity menu listing RPA activities for interacting with the mobile computing device;
configure the first RPA activity further according to a user's selection from the first activity menu; and
configure the second RPA activity further according to a user's selection from the second activity menu.

17. The computer system of claim 10 , wherein the at least one hardware processor is further configured to, in response to the second user input, update the model GUI to show a result of the second user input.

18. The computer system of claim 10 , wherein the second RPA activity comprises executing a user gesture for interacting with the mobile computing device.

19. A non-transitory computer-readable medium storing instructions which, when executed by at least one hardware processor of a computer system, cause the computer system to:
display a desktop graphical user interface (GUI) and a model GUI on a screen of the computer system, wherein the model GUI mimics a display of a mobile computing device targeted for robotic process automation (RPA);
construct an RPA workflow comprising a first RPA activity for interacting with a desktop computer and a second RPA activity for interacting with the mobile computing device, wherein constructing the RPA workflow comprises:
receiving a first user input to the desktop GUI,
in response to the first user input, configuring the first RPA activity according to the first user input,
receiving a second user input to the model GUI, and
in response to the second user input, configuring the second RPA activity according to the second user input; and

generate at least one robot specification according to the RPA workflow, the at least one robot specification instructing a first RPA robot to carry out the first RPA activity on the desktop computer and instructing a second RPA robot to carry out the second RPA activity on the mobile computing devi

Assignees

• Uipath Inc

Inventors

• Mayer Christian
• Weishaar Gerd
• Cucosel Bogdan

Classifications

• G06F9/452 Primary

  Remote windowing, e.g. X-Window System, desktop virtualisation (protocols for virtual reality H04L67/131) · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F9/547

  Remote procedure calls [RPC]; Web services · CPC title

• G05B2219/40391

  Human to robot skill transfer · CPC title

• G05B2219/40392

  Programming, visual robot programming language · CPC title