Robot process automation apparatus and method for detecting changes thereof

Patent metadata

Field	Value
Publication number	US-11279040-B2
Application number	US-201916664494-A
Country	US
Kind code	B2
Filing date	Oct 25, 2019
Priority date	Oct 24, 2019
Publication date	Mar 22, 2022
Grant date	Mar 22, 2022

Abstract

According to aspects of the present disclosure, a change detection method for ROBOT process automation (RPA) performed by a computer device includes obtaining object information of a target object from first working environment information at a first time, and determining a comparison object detected in second working environment information at a second time as an object corresponding to the target object based on the obtained object information of the target information. Thus, even if an error occurs due to a change in a working environment, an RPA apparatus can operate normally by correcting the error by itself without the need for a user to recreate a new bot or modify the existing bot. Also, even if the position or shape of an object referred to by RPA in a screen is changed, the RPA apparatus can accurately find an object to manipulate based on surrounding information and perform RPA normally.

First claim

What is claimed is:

1. A change detection method for ROBOT process automation (RPA) performed by a computer device, the method comprising:
obtaining object information of a target object from first working environment information at a first time; and
determining a comparison object detected in second working environment information at a second time as an object corresponding to the target object based on the obtained object information of the target information,
wherein the determining of the comparison object as the object corresponding to the target object comprises:
searching for an object structure of the comparison object in the second working environment information and determining the comparison object as a candidate object according to the search result; and
calculating similarity of the candidate object by generating object identification text of the candidate object based on attribute information of the candidate object, the calculating of the similarity comprising:
generating the object identification text by processing the attribute information of the candidate object;
generating a vector by converting the object identification text, the generating of the vector comprising generating an eigenvector of the candidate object by converting the object identification text through text embedding; and
calculating the similarity of the candidate object by calculating similarity between vectors using the generated vector.

2. The method of claim 1 , wherein the determining of the comparison object as the candidate object comprises:
obtaining the object structure of the comparison object from the second working environment information;
determining whether the object structure of the comparison object is the same or similar to an object structure of the target object; and
determining the comparison object as the candidate object if it is determined that the object structure of the comparison object is the same or similar to the object structure of the target object.

3. The method of claim 1 , wherein the object structure is information about a hierarchical relationship between nodes identified around the comparison object in a tree structure of objects in the second working environment information.

4. The method of claim 3 , wherein the object structure comprises tag information of the comparison object, tag information of a parent object of the comparison object, tag information of a child object of the comparison object, or tag information of a neighbor object of the comparison object.

5. The method of claim 1 , wherein the generating of the object identification text comprises:
filtering out some of the attribute information of the candidate object;
configuring texts, which remain after the filtering of the attribute information, in units of corpora; and
converting the texts configured in units of corpora into a predetermined language.

6. The method of claim 1 , wherein the eigenvector of the candidate object has a size and direction similar to those of an eigenvector of the target object as the object identification text of the candidate object is similar to object identification text of the target object.

7. The method of claim 1 , wherein the determining of the comparison object as the object corresponding to the target object further comprises determining the candidate object as the object corresponding to the target object if the candidate object has a higher similarity value than other candidate objects.

8. The method of claim 1 , further comprising providing object information of the comparison object to update the object information of the target object.

9. A change detection method for ROBOT process automation (RPA) performed by a computer device, the method comprising:
executing a bot for RPA;
determining whether an operation error of the bot has occurred;
detecting changes in an object referred to by the bot if it is determined that the operation error has occurred; and
updating object information of the object based on the change detection result,
wherein the detecting of the changes in the object comprises:
obtaining object information of a target object from first working environment information at a first time; and
determining a comparison object detected in second working environment information at a second time as an object corresponding to the target object based on the obtained object information of the target information,
wherein the determining of the comparison object as the object corresponding to the target object comprises:
searching for an object structure of the comparison object in the second working environment information and determining the comparison object as a candidate object according to the search result; and
calculating similarity of the candidate object by generating object identification text of the candidate object based on attribute information of the candidate object the calculating of the similarity comprising:
generating the object identification text by processing the attribute information of the candidate object;
generating a vector by converting the object identification text, the generating of the vector comprising generating an eigenvector of the candidate object by converting the object identification text through text embedding; and
calculating the similarity of the candidate object by calculating similarity between vectors using the generated vector.

10. A change detection apparatus for ROBOT process automation (RPA) performed by a computer device, the apparatus comprising:
a memory which loads a computer program comprising one or more instructions; and
a processor which executes the instructions,
wherein the computer program comprises:
an RPA performing module which creates and executes a bot for RPA; and
a change detection module which detects changes in an object referred to by an event having an error when the error occurs during execution of the bot,
wherein the change detection module determines a comparison object detected in working environment information at a time when the error occurred as the object referred to by the event based on object information of the object referred to by the event; and
wherein the change detection module is configured to detect the changes in the object by:
obtaining object information of a target object from first working environment information at a first time; and
determining a comparison object detected in second working environment information at a second time as an object corresponding to the target object based on the obtained object information of the target information, wherein the determining of the comparison object as the object corresponding to the target object comprises:
searching for an object structure of the comparison object in the second working environment information and determining the comparison object as a candidate object according to the search result; and
calculating similarity of the candidate object by generating object identification text of the candidate object based on attribute information of the candidate object, the calculating of the similarity comprising:
generating the object identification text by processing the attribute information of the candidate object;
generating a vector by converting the object identification text, the generating of the vector comprising generating an eigenvector of the candidate object by converting the object identification text through text embedding; and
calculating the similarity of the candidate object by calculating similarity between vectors using the generated vector.

11. The apparatus of claim 10 , wherein the RPA performing module comprises:
a bot modeling unit which crea

Assignees

• Samsung Sds Co Ltd

Inventors

• An Kyung Jun
• Jang Sung Ho
• Cho Jun Hee
• Jo Jong Yoon

Classifications

• B25J9/161

  Hardware, e.g. neural networks, fuzzy logic, interfaces, processor · CPC title

• G05B19/04 Primary

  Program control other than numerical control, i.e. in sequence controllers or logic controllers (G05B19/418 takes precedence) · CPC title

• G06F2201/81

  Threshold · CPC title

• B25J9/1697 Primary

  Vision controlled systems · CPC title

• B25J9/1692

  Calibration of manipulator · CPC title