Evaluating user responses based on bootstrapped knowledge acquisition from a limited knowledge domain

Patent metadata

Field	Value
Publication number	US-10832591-B2
Application number	US-201916266676-A
Country	US
Kind code	B2
Filing date	Feb 4, 2019
Priority date	Nov 11, 2016
Publication date	Nov 10, 2020
Grant date	Nov 10, 2020

Abstract

Mechanisms for training a human user to perform an operation and provided. The mechanisms generate a domain specific knowledge base comprising a set of entities and corresponding domain specific attributes and expand the domain specific knowledge base to include values for the domain specific attributes through an automated bootstrap learning process that performs natural language processing and analysis of natural language content using a set of pre-condition annotated action terms, thereby generating an expanded domain specific knowledge base. The mechanisms evaluate an input from another device identifying an action associated with an entity in the set of entities, based on a retrieved domain specific attribute value and the retrieved pre-condition annotation from the expanded domain specific knowledge base. The mechanisms output a notification to a user computing device indicating whether the input is correct or incorrect to thereby train a user associated with the user computing device.

First claim

What is claimed is:

1. A method, in a data processing system comprising a processor and a memory accessible by the processor, for training a human user to perform an operation, the method comprising:
generating, by the data processing system, a domain specific knowledge base comprising a set of entities and corresponding domain specific attributes;
expanding, by the data processing system, the domain specific knowledge base to include values for the domain specific attributes through an automated bootstrap learning process that performs natural language processing and analysis of natural language content using a set of pre-condition annotated action terms, thereby generating an expanded domain specific knowledge base;
obtaining, by the data processing system, an input from another device identifying an action associated with an entity in the set of entities;
retrieving, by the data processing system, from the expanded domain specific knowledge base, a domain specific attribute value for the entity identified in the input and a pre-condition annotation associated with the action identified in the input;
evaluating, by the data processing system, a correctness or incorrectness of the input based on the retrieved domain specific attribute value and the retrieved pre-condition annotation;
outputting, by the data processing system, a notification to a user computing device indicating whether the input is correct or incorrect to thereby train a user associated with the user computing device; and
maintaining, by the data processing system, a temporal ordering of actions performed with regard to the entity identified in the input, wherein evaluating the correctness or incorrectness of the input based on the retrieved domain specific attribute value and the retrieved pre-condition annotation is further performed based on a current state of the entity as determined from a post-condition annotation associated with an action term corresponding to a last performed action in the temporal ordering of actions performed with regard to the entity.

2. The method of claim 1 , wherein the bootstrap learning process associates a value with a domain specific attribute of an entity, in the set of entities, based on a pre-condition value of a pre-condition annotation of an instance of an action term, in the set of pre-condition annotated action terms, that is correlated with the entity in the natural language content.

3. The method of claim 2 , wherein the another device is the user computing device, and wherein the input is an input from the user in response to a training inquiry presented to the user based on the expanded domain specific knowledge base.

4. The method of claim 1 , wherein the another device is a sensor device that detects an input from a user representative of the action and the entity.

5. The method of claim 4 , wherein the sensor device is at least one of an image capturing device or an audio capturing device, and wherein the evaluation of the correctness or incorrectness of the input comprises performing at least one of image analysis or audio analysis to identify the action and the entity.

6. The method of claim 1 , wherein evaluating, by the data processing system, a correctness or incorrectness of the input based on the retrieved domain specific attribute value and the retrieved pre-condition annotation further comprises:
evaluating a post-condition of a previous action performed with regard to the entity to determine a new value for the domain specific attribute indicating a result of the performance of the previous action on the entity; and
correlating the new value for the domain specific attribute with the retrieved pre-condition annotation.

7. The method of claim 1 , wherein pre-condition annotations of the set of pre-condition annotated action terms specify, for each pre-condition annotated action term, a required value of a domain specific attribute of an entity upon which an action corresponding to the pre-condition annotated action term may be correctly performed.

8. The method of claim 1 , wherein the entities in the set of entities are ingredients for cooking recipes, and wherein the action terms in the set of pre-condition annotated action terms are actions that are able to be performed on the ingredients, and wherein, for each pre-condition annotation action term in the set of pre-condition action terms, a pre-condition annotation of the pre-condition annotated action term specifies a state of matter of an ingredient required in order for a corresponding action to be correctly performed on the ingredient.

9. The method of claim 1 , wherein the entities in the set of entities and the pre-condition annotated action terms in the set of pre-condition annotated action terms are associated with a domain in which temporally ordered tasks are to be followed to complete an operation.

10. A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a data processing system, causes the data processing system to:
generate a domain specific knowledge base comprising a set of entities and corresponding domain specific attributes;
expand the domain specific knowledge base to include values for the domain specific attributes through an automated bootstrap learning process that performs natural language processing and analysis of natural language content using a set of pre-condition annotated action terms, thereby generating an expanded domain specific knowledge base;
obtain an input from another device identifying an action associated with an entity in the set of entities;
retrieve, from the expanded domain specific knowledge base, a domain specific attribute value for the entity identified in the input and a pre-condition annotation associated with the action identified in the input;
evaluate a correctness or incorrectness of the input based on the retrieved domain specific attribute value and the retrieved pre-condition annotation;
output a notification to a user computing device indicating whether the input is correct or incorrect to thereby train a user associated with the user computing device; and
maintain a temporal ordering of actions performed with regard to the entity identified in the input, wherein evaluating the correctness or incorrectness of the input based on the retrieved domain specific attribute value and the retrieved pre-condition annotation is further performed based on a current state of the entity as determined from a post-condition annotation associated with an action term corresponding to a last performed action in the temporal ordering of actions performed with regard to the entity.

11. The computer program product of claim 10 , wherein the bootstrap learning process associates a value with a domain specific attribute of an entity, in the set of entities, based on a pre-condition value of a pre-condition annotation of an instance of an action term, in the set of pre-condition annotated action terms, that is correlated with the entity in the natural language content.

12. The computer program product of claim 11 , wherein the another device is the user computing device, and wherein the input is an input from the user in response to a training inquiry presented to the user based on the expanded domain specific knowledge base.

13. The computer program product of claim 10 , wherein the another device is a sensor device that detects an input from a user representative of the action and the entity.

14. The computer program product of claim 13 , wherein the sensor device is at least one of an image cap

Assignees

• IBM

Inventors

• Byron Donna K
• Johnson Benjamin L
• Tecuci Dan G

Classifications

• G06N20/00

  Machine learning · CPC title

• G06N5/022

  Knowledge engineering; Knowledge acquisition · CPC title

• G09B19/0092 Primary

  Nutrition · CPC title

• G09B5/02

  with visual presentation of the material to be studied, e.g. using film strip · CPC title

• G09B7/02

  of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student · CPC title