Natural question generation via reinforcement learning based graph-to-sequence model

Patent metadata

Field	Value
Publication number	US-11816136-B2
Application number	US-202217971635-A
Country	US
Kind code	B2
Filing date	Oct 23, 2022
Priority date	Jan 2, 2020
Publication date	Nov 14, 2023
Grant date	Nov 14, 2023

Abstract

For a passage text and a corresponding answer text, perform a word-level soft alignment to obtain contextualized passage embeddings and contextualized answer embeddings, and a hidden level soft alignment on the contextualized passage embeddings and the contextualized answer embeddings to obtain a passage embedding matrix. Construct a passage graph of the passage text based on the passage embedding matrix, and apply a bidirectional gated graph neural network to the passage graph until a final state embedding is determined, during which intermediate node embeddings are fused from both incoming and outgoing edges. Obtain a graph-level embedding from the final state embedding, and decode the final state embedding to generate an output sequence word-by-word. Train a machine learning model to generate at least one question corresponding to the passage text and the answer text, by evaluating the output sequence with a hybrid evaluator combining cross-entropy evaluation and reinforcement learning evaluation.

First claim

What is claimed is:

1. A method comprising:
obtaining contextualized passage embeddings and contextualized answer embeddings for a text pair;
obtaining a passage embedding matrix;
constructing a corresponding passage graph based on said passage embedding matrix;
applying a bidirectional gated graph neural network to said corresponding passage graph until a final state embedding is determined, during which application intermediate node embeddings are fused from both incoming and outgoing edges of said graph;
obtaining a graph-level embedding from said final state embedding;
decoding said final state embedding to generate an output sequence; and
training a machine learning model to generate at least one question corresponding to said text pair by evaluating said output sequence.

2. The method of claim 1 , further comprising using said trained machine learning module to respond to a user query.

3. The method of claim 2 , wherein said user query pertains to information technology, further comprising configuring at least one information technology asset in accordance with said response.

4. The method of claim 2 , wherein training said machine learning model by evaluating said output sequence comprises optimizing a reward function combining an evaluation metric reward function and a semantic reward function.

5. The method of claim 2 , wherein said training comprises initial training with cross-entropy loss and fine-tuning to optimize a scaling factor combining cross-entropy loss and reinforcement learning loss.

6. A non-transitory computer readable medium comprising computer executable instructions which when executed by a computer cause the computer to perform a method of:
obtaining contextualized passage embeddings and contextualized answer embeddings for a text pair;
obtaining a passage embedding matrix;
constructing a corresponding passage graph based on said passage embedding matrix;
applying a bidirectional gated graph neural network to said corresponding passage graph until a final state embedding is determined, during which application intermediate node embeddings are fused from both incoming and outgoing edges of said graph;
obtaining a graph-level embedding from said final state embedding;
decoding said final state embedding to generate an output sequence; and
training a machine learning model to generate at least one question corresponding to said text pair by evaluating said output sequence.

7. The non-transitory computer readable medium of claim 6 , wherein said method further comprises using said trained machine learning module to respond to a user query.

8. The non-transitory computer readable medium of claim 7 , wherein said user query pertains to information technology, wherein said method further comprises facilitating configuring at least one information technology asset in accordance with said response.

9. The non-transitory computer readable medium of claim 7 , wherein training said machine learning model by evaluating said output sequence comprises optimizing a reward function combining an evaluation metric reward function and a semantic reward function.

10. The non-transitory computer readable medium of claim 7 , wherein said training comprises initial training with cross-entropy loss and fine-tuning to optimize a scaling factor combining cross-entropy loss and reinforcement learning loss.

11. An apparatus comprising:
a memory;
a non-transitory computer readable medium comprising computer executable instructions; and
at least one processor, coupled to said memory and said non-transitory computer readable medium, and operative to execute said instructions to be operative to:
obtain contextualized passage embeddings and contextualized answer embeddings for a text pair;
obtain a passage embedding matrix;
construct a corresponding passage graph based on said passage embedding matrix;
apply a bidirectional gated graph neural network to said corresponding passage graph until a final state embedding is determined, during which application intermediate node embeddings are fused from both incoming and outgoing edges of said graph;
obtain a graph-level embedding from said final state embedding;
decode said final state embedding to generate an output sequence; and
train a machine learning model to generate at least one question corresponding to said text pair by evaluating said output sequence.

12. The apparatus of claim 11 , wherein said at least one processor is further operative to execute said instructions to use said trained machine learning module to respond to a user query.

13. The apparatus of claim 12 , wherein said user query pertains to information technology, wherein said at least one processor is further operative to execute said instructions to facilitate configuring at least one information technology asset in accordance with said response.

14. The apparatus of claim 12 , wherein training said machine learning model by evaluating said output sequence comprises optimizing a reward function combining an evaluation metric reward function and a semantic reward function.

15. The apparatus of claim 12 , wherein said training comprises initial training with cross-entropy loss and fine-tuning to optimize a scaling factor combining cross-entropy loss and reinforcement learning loss.

Assignees

• IBM
• Rensselaer Polytech Inst

Inventors

• Wu Lingfei
• Chen Yu
• Zaki Mohammed J

Classifications

• G06N3/0475

  Generative networks · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/092

  Reinforcement learning · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title