Method, system, electronic device and storage medium for clarification question generation

Patent metadata

Field	Value
Publication number	US-11475225-B2
Application number	US-202217698722-A
Country	US
Kind code	B2
Filing date	Mar 18, 2022
Priority date	Mar 22, 2021
Publication date	Oct 18, 2022
Grant date	Oct 18, 2022

Abstract

A method, a device and electronic device for clarification question generation are provided in one or more embodiments of this disclosure. The method includes: extracting entity information from a fuzzy context input by a user; inputting the fuzzy context into a template generating module of a pre-built CQG neural network model so as to obtain a clarification question template; inputting the entity information into an entity rendering module of the CQG neural network model so as to obtain at least one entity phrase; and generating a clarification question for a fuzzy question based on the clarification question template and the at least one entity phrase for presenting to the user.

First claim

What is claimed is:

1. A method for clarification question generation executed by a processor, comprising:
extracting entity information from a fuzzy context in response to receiving the fuzzy context including historical questions, historical answers and fuzzy questions input by a user via an input interface;
generating a Clarification Question Generation (CQG) neural network model by adding a layered Transformer mechanism and a pointer generator mechanism into a coarse-to-fine model;
inputting the fuzzy context into a template generating module of the CQG neural network model so as to obtain a clarification question template;
inputting the entity information into an entity rendering module of the CQG neural network model so as to obtain at least one entity phrase; and
applying the CQG neural network model to generate a clarification question with an enhanced recall rate and consistency for a fuzzy question based on the clarification question template and the at least one entity phrase for presenting to the user; wherein
the clarification question comprises irrelevant words, and the irrelevant words are eliminated from the clarification question via the pointer generator mechanism, resulting in the enhanced recall rate and consistency, and
the CQG neural network model is trained with a collaborative training paradigm, which combines an auxiliary task based on self-supervised learning with a downstream CQG task for end-to-end training.

2. The method according to claim 1 , wherein the template generating module comprises a template generating encoder and a template generating decoder;
the template generating encoder comprises a first layered Transformer encoder layer;
the template generating decoder comprises a first layered Transformer decoder layer and a first pointer generator; and
inputting the fuzzy context into the template generating module so as to obtain the clarification question template comprises: forming a first word embedding with a predetermined length and a predetermined dimension based on the fuzzy context;
converting, the first layered Transformer encoder layer, the first word embedding into a first layered context representation; and
generating the clarification question template based on the first layered context representation by the first layered Transformer decoder layer and the first pointer generator.

3. The method according to claim 2 , wherein generating the clarification question template by the first layered Transformer decoder layer and the first pointer generator comprises:
generating a first predictive output representation based on the first layered context representation and calculating a first probability of generating words from a preset vocabulary by using a softmax function according to the first predictive output representation, by the first layered Transformer decoder layer;
taking the first pointer generator as a first soft switch to select to copy the first word from the fuzzy context according to an attention distribution output by a last decoder layer in the first layered Transformer decoder layer, or to generate a second word from the vocabulary according to the first final probability obtained from the first probability; and
generating the clarification question template based on the first word and/or the second word.

4. The method according to claim 3 , wherein the entity rendering module comprises an entity rendering encoder and an entity rendering decoder;
the entity rendering encoder comprises a second layered Transformer encoder layer;
the entity rendering decoder comprises a second layered Transformer decoder layer and a second pointer generator;
inputting the entity information into the entity rendering module so as to obtain the at least one entity phrase comprises:
forming a second word embedding with the predetermined length and the predetermined dimension based on the entity information;
converting, by the second layered Transformer encoder layer, the second word embedding into a second layered context representation; and
generating the at least one entity phrase based on the second layered context representation by the second layered Transformer decoder layer and the second pointer generator.

5. The method according to claim 4 , wherein generating the at least one entity phrase by the second layered Transformer decoder layer and the second pointer generator comprises:
generating a second predictive output representation based on the second layered context representation and calculating a second probability of generating words from the vocabulary by using a softmaxfunction according to the second predictive output representation, by the second layered Transformer decoder layer;
taking the second pointer generator as a second soft switch to select to copy the third word from the entity information according to an attention distribution output by a last decoder layer in the second layered Transformer decoder layer, or to generate a forth word from the vocabulary according to the second final probability obtained from the second probability; and
generating the at least one entity phrase based on the third word and/or the fourth word.

6. The method according to claim 5 , wherein training instances are randomly sampled from a mixed training set of the auxiliary task based on self-supervised learning and the downstream CQG task to input to the CQG neural network model to train for a minimum total loss.

7. The method according to claim 6 , wherein the fuzzy context comprises a historical question, a historical answer and a fuzzy question; the entity information comprises an entity name, an entity type and an entity description;
the auxiliary task based on self-supervised learning comprises a dialogue history prediction auxiliary task and an entity name prediction auxiliary task;
a dialog history template is generated by screening the entity name according to the fuzzy question in the dialog history prediction auxiliary task;
at least one predicted entity name is generated according to the entity type and the entity description in the entity name prediction auxiliary task; and
in the dialog history prediction auxiliary task, a predicted history question and a predicted history question answer are generated according to the dialog history template and the prediction entity name, and finally a dialog history is generated according to the predicted history and the predicted history answer.

8. An electronic device, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of claim 1 when executing the computer program.

9. The electronic device of claim 8 , wherein the template generating module comprises a template generating encoder and a template generating decoder;
the template generating encoder comprises a first layered Transformer encoder layer;
the template generating decoder comprises a first layered Transformer decoder layer and a first pointer generator; and
inputting the fuzzy context into the template generating module so as to obtain the clarification question template comprises: forming a first word embedding with a predetermined length and a predetermined dimension based on the fuzzy context;
converting, the first layered Transformer encoder layer, the first word embedding into a first layered context representation; and
generating the clarification question template based on the first layered context representation by the first layered Transformer decoder layer and the first pointer generator.

10. The electronic device of claim 9 , wherein generating the clarification question template by the first layered Transformer decod

Assignees

• National Univ Of Defense Technology

Inventors

• Chen Honghui
• Shao taihua
• Cai Fei
• Shu Zhen
• Chen Wanyu
• Yan Jingjing
• Chen Tao
• Luo Aimin
• Zhang Mengmeng

Classifications

• G06N3/043

  based on fuzzy logic, fuzzy membership or fuzzy inference, e.g. adaptive neuro-fuzzy inference systems [ANFIS] · CPC title

• G06F40/40 Primary

  Processing or translation of natural language (natural language analysis G06F40/20; semantic analysis G06F40/30) · CPC title

• G06F40/295 Primary

  Named entity recognition · CPC title

• G06F40/253

  Grammatical analysis; Style critique · CPC title

• G06F40/186

  Templates · CPC title