Dynamic learning method and system for robot, robot and cloud server

Patent metadata

Field	Value
Publication number	US-11580454-B2
Application number	US-202016815840-A
Country	US
Kind code	B2
Filing date	Mar 11, 2020
Priority date	Sep 12, 2017
Publication date	Feb 14, 2023
Grant date	Feb 14, 2023

Abstract

A dynamic learning method for a robot includes a training and learning mode. The training and learning mode includes the following steps: dynamically annotating a belonging and use relationship between an object and a person in a three-dimensional environment to generate an annotation library; acquiring a rule library, and establishing a new rule and a new annotation by means of an interactive demonstration behavior based on the rule library and the annotation library; and updating the new rule to the rule library and updating the new annotation to the annotation library when it is determined that the established new rule is not in conflict with rules in the rule library and the new annotation is not in conflict with annotations in the annotation library.

First claim

What is claimed is:

1. A dynamic learning method for a robot, comprising a training and learning mode; wherein the training and learning mode comprises the following steps:
dynamically annotating a belonging and use relationship between an object and a person in a three-dimensional environment to generate an annotation library;
acquiring a rule library, and establishing a new rule and a new annotation by means of an interactive demonstration behavior based on the rule library and the annotation library; and
updating the new rule to the rule library when it is determined that the established new rule is not in conflict with rules in the rule library;
the dynamic learning method for a robot further comprising a working mode; wherein the working mode comprises the following steps:
receiving an audio instruction from a user, and identifying the audio instruction;
carrying out task planning based on the identified audio instruction and the rule library; and
judging whether the task planning is in conflict with a rule, and establishing an interrogative interaction with the user if the task planning is in conflict with a rule;
the interrogative interaction comprises:
an interrogative interaction for continuously carrying out the task planning, wherein the conflicted rule is ignored when the user selects to continuously carry out the interrogative interaction;
an interrogative interaction for canceling the task planning, wherein the operating mode is switched when the user selects to cancel the interrogative interaction; and
an interrogative interaction for entering the training and learning mode.

2. The method according to claim 1 , further comprising:
updating the new annotation to the annotation library when it is determined that the new annotation is not in conflict with annotations in the annotation library; and
completing a task and an interaction designated by a user based on the annotation library and the rule library.

3. The method according to claim 1 , wherein the step of dynamically annotating the belonging and use relationship between the object and the person in the three-dimensional environment comprises:
calling a robot three-dimensional environment semantic map based on machine vision and natural language understanding; and
acquiring a semantic map in a current scenario or the interactive demonstration behavior, identifying whether the semantic map in the current scenario or the interactive demonstration behavior comprises a new belonging and use relationship, parsing and annotating the new belonging and use relationship, and storing the new belonging and use relationship to the annotation library.

4. The method according to claim 3 , wherein each rule and interactive demonstration behavior in the rule library comprise four elements: subject, object, action to be performed and whether the action is allowed; and the rule library comprises a default scenario rule and a learning rule.

5. An electronic device, comprising at least one processor; and a memory communicably connected to the at least one processor: wherein
the memory stores instructions executable by the at least one processor, wherein, the instructions, when being executed by the at least one processor, cause the at least one processor to perform the steps of:
the electronic device further training and learning mode: wherein the training and learning mode comprises the following steps:
dynamically annotating a belonging and use relationship between an object and a person in a three-dimensional environment to generate an annotation library;
acquiring a rule library, and establishing new rule and a new annotation by means of an interactive demonstration behavior based on the rule library and the annotation library; and
updating the new rule to the rule library when it is determined that the established new rule is not in conflict with rules in the rule library;
the electronic device further comprising a working mode: wherein the working mode comprises the following steps:
receiving an audio instruction from a user, and identifying the audio instruction;
carrying out task planning based on the identified audio instruction and the rule library; and
judging whether the task planning is in conflict with a rule, and establishing an interrogative interaction with the user if the task planning is in conflict with a rule;
the interrogative interaction comprises:
an interrogative interaction for continuously carrying out the task planning, wherein the conflicted rule is ignored when the user selects to continuously carry out the interrogative interaction;
an interrogative interaction for canceling the task planning, wherein the operating mode is switched when the user elects to cancel the interrogative interaction; and
an interrogative interaction for entering the training and learning mode.

6. The electronic device according to claim 5 , wherein, the instructions, when being executed by the at least one processor, cause the at least one processor to further perform the steps of:
updating the new annotation to the annotation library when it is determined that the new annotation is not in conflict with annotations in the annotation library; and
completing a task and an interaction designated by a user based on the annotation library and the rule library.

7. The electronic device according to claim 5 , wherein the step of dynamically annotating the belonging and use relationship between the object and the person in the three dimensional environment comprises:
calling a robot three-dimensional environment semantic map based on machine vision and natural language understanding; and
acquiring a semantic map in a current scenario or the interactive demonstration behavior, identifying whether the semantic map in the current scenario or the interactive demonstration behavior comprises a new belonging and use relationship, parsing and annotating the new belonging and use relationship, and storing the new belonging and use relationship to the annotation library.

8. The electronic device according to claim 7 , wherein each rule and interactive demonstration behavior in the rule library comprise four elements: subject, object, action to be performed and whether the action is allowed; and the rule library comprises a default scenario rule and a learning rule.

9. A non-transitory computer-readable storage medium, wherein the computer readable storage medium stores computer executable instructions, which, when being executed by at least one processor, may cause the at least one processor to perform the steps of:
the electronic device further training and learning mode: wherein the training and learning mode comprises the following steps:
dynamically annotating a belonging and use relationship between an object and a person in a three-dimensional environment to generate an annotation library;
acquiring a rule library, and establishing a new rule and a new annotation by means of an interactive demonstration behavior based on the rule library and the annotation library; and
updating the new rule to the rule library when it is determined that the established new rule is not in conflict with rules in the rule library;
the electronic device further comprising a working mode; wherein the working mode comprises the following steps:
receiving an audio instruction from a user, and identifying the audio instruction;
carrying out task planning based on the identified audio instruction and the rule library; and
judging whether the task planning is in conflict with a rule, and establishing an interrogative interaction with the user if the task planning is in conflict with a rule;
the interrogative interaction comprises:
an interrogative interaction for con

Assignees

• Cloudminds Shenzhen Robotics Systems Co Ltd
• Cloudminds Robotics Co Ltd

Inventors

• Zhang Zhanchao

Classifications

• G06N5/025 Primary

  Extracting rules from data · CPC title

• G06N5/022

  Knowledge engineering; Knowledge acquisition · CPC title

• G05B2219/39299

  Learn forward dynamics · CPC title

• B25J11/0005 Primary

  Manipulators having means for high-level communication with users, e.g. speech generator, face recognition means · CPC title

• G06N3/008

  based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour · CPC title