Question answering information completion using machine reading comprehension-based process

Patent metadata

Field	Value
Publication number	US-12112135-B2
Application number	US-202117449303-A
Country	US
Kind code	B2
Filing date	Sep 29, 2021
Priority date	Sep 29, 2021
Publication date	Oct 8, 2024
Grant date	Oct 8, 2024

Abstract

An approach is provided for optimizing a feedback-type question answering process. A training set is constructed to detect missing information of a question. A natural language generation model is trained using the missing information. The natural language generation model is executed to generate a rhetorical question. A response to the rhetorical question is combined with the question to generate an input to a language processor. A new question is generated. The new question is applied to a document library. A final answer is generated.

First claim

What is claimed is:

1. A computer system comprising:
a central processing unit (CPU);
a memory coupled to the CPU; and
one or more computer readable storage media coupled to the CPU, the one or more computer readable storage media collectively containing instructions that are executed by the CPU via the memory to cause the processor to implement a question answering system process, comprising:
constructing, by the processor, a training set to detect missing information of a question received by the processor, the question semantically related to a source of information in a document library;
identifying a plurality of components of the question, the components including information terms of the question;
generating a plurality of masked questions from the question, including, for each of the masked questions, masking one component of the plurality of components of the question at a time until each of the plurality of components is masked to generate the masked question;
applying a reading comprehension algorithm to the each masked question to generate new training data;
generating the new training data from a combination of the source of information, the masked component, the question, and an answer assessment generated from a comparison of the source of information and the component of the question that is masked;
receiving, by a reinforcement learning (RL) system, the new training data to predict the missing information of the question;
training, by the processor, a natural language generation model using the missing information, including, in response to the generating the answer assessment, selecting words about which another question is generated for clarifying the question;
executing, by the processor, the natural language generation model to generate the other question clarifying the question using the selected words;
combining, by the processor, a response to the other question and the question to generate an input to a language processor;
generating, by the language processor, a new question;
applying the new question to a document library; and
generating a final answer.

2. The computer system of claim 1 , wherein constructing the training set includes the computer executing the reading comprehension dataset to according to the reinforcement learning (RL) system to predict the missing information in the question.

3. The computer system of claim 2 , wherein the training set includes a Stanford Question Answering Dataset (SQUAD)-type comprehension dataset, and wherein a new dataset is constructed from the SQUAD-type comprehension dataset.

4. The computer system of claim 3 , wherein the new dataset is further generated according to a reinforcement learning model to detect the missing information.

5. The computer system of claim 1 , wherein the language processor includes executes combination of a Generative Pre-trained Transformer (GPT)-3 model and a text style transfer (TST) algorithm to generate the new question.

6. The computer system of claim 5 , wherein the combination of the GPT-3 model and the TST algorithm are performed on contents of the document library.

7. The computer system of claim 1 , wherein the document library is part of a knowledge base.

8. The computer system of claim 1 , wherein constructing the training set includes:
applying, by the processor, a data mask to the each component of the question to cover or redact the information terms of the question;
applying, by the processor, the reading comprehension algorithm to the question with the covered or information terms; and
searching, by the processor, for information in the document library semantically related to the original question to generate the new training data.

9. The computer system of claim 8 , wherein the new training data is generated by a predetermined set of rules establishing a reward score of the answer assessment established by conditions for masking the terms, so that that the new training data includes a combination of the masked word, the question, the reward, the information, and the answer, and wherein the new training data is applied to a reinforcement learning technique of the RL system.

10. A feedback-type question answering method, comprising:
receiving, by a data processing system, a question from a user, the data processing system comprising at least one processor and at least one memory, the at least one memory comprising instructions which are executed by the at least one processor;
constructing, by the at least one processor, a training set to detect missing information of the question received by the at least one processor, the question semantically related to a source of information in a document library;
identifying a plurality of components of the question, the components including information terms of the question;
generating a plurality of masked questions from the question, including, for each of the masked questions, masking one component of the plurality of components of the question at a time until each of the plurality of components is masked to generate the masked question;
applying a reading comprehension algorithm to the each masked question to generate new training data;
generating the new training data from a combination of the source of information, the masked component, the question, and an answer assessment generated from a comparison of the source of information and the component of the question that is masked;
receiving, by a reinforcement learning (RL) system, the new training data to predict the missing information of the question;
training, by the at least one processor, a natural language generation model using the missing information, including, in response to the generating the answer assessment, selecting words about which another question is generated for clarifying the question;
executing, by the at least one processor, the natural language generation model to generate the other question clarifying the question using the selected words;
combining, by the at least one processor, a response to the other question and the question to generate an input to a language processor;
generating, by the language processor, a new question;
applying the new question to a document library; and
generating a final answer.

11. The feedback-type question answering method of claim 10 , wherein constructing the training set includes executing the reading comprehension dataset to according to the reinforcement learning (RL) system to predict the missing information in the question.

12. The feedback-type question answering method of claim 11 , wherein the training set includes a Stanford Question Answering Dataset (SQUAD)-type comprehension dataset, and wherein a new dataset is constructed from the SQUAD-type comprehension dataset.

13. The feedback-type question answering method of claim 12 , wherein the new dataset is further generated according to a reinforcement learning model to detect the missing information.

14. The feedback-type question answering method of claim 10 , wherein the language processor includes executes a Generative Pre-trained Transformer (GPT)- 3 model and a text style transfer (TST) algorithm to generate the new question.

15. The feedback-type question answering method of claim 14 , wherein the GPT-3 model and the TST algorithm are performed on contents of the document library.

16. The feedback-type question answering method of claim 10 , wherein the document library is part of a knowledge base.

17. The feedback-type question answering method of claim 10 , wherein constructing the data set includes:
app

Assignees

• IBM

Inventors

• Yuan Zhong Fang
• Liu Tong
• Gao Chen
• Yang Xiang Yu

Classifications

• G06F18/2148

  characterised by the process organisation or structure, e.g. boosting cascade · CPC title

• G06N5/04

  Inference or reasoning models · CPC title

• G06N3/096

  Transfer learning · CPC title

• G06N3/092

  Reinforcement learning · CPC title

• G06N3/045

  Combinations of networks · CPC title