Neural models for key phrase detection and question generation

Patent metadata

Field	Value
Publication number	US-2021134173-A1
Application number	US-202117143303-A
Country	US
Kind code	A1
Filing date	Jan 7, 2021
Priority date	Aug 3, 2017
Publication date	May 6, 2021
Grant date	—

Abstract

A method, system, and storage device storing a computer program, for generating questions based on provided content, such as, for example, a document having words. The method comprises automatically estimating the probability of interesting phrases in the provided content, and generating a question in natural language based on the estimating. In one example embodiment herein, the estimating includes predicting the interesting phrases as answers, and the estimating is performed by a neural model. The method further comprises conditioning a question generation model based on the interesting phrases predicted in the predicting, the question generation model generating the question. The method also can include training the neural model. In one example, the method further comprises identifying start and end locations of the phrases in the provided content, and the identifying includes performing a dot product attention mechanism parameterizing a probability distribution.

First claim

1 - 20 . (canceled)

21 . A method executed by a computer processor, for generating questions based on provided content, the method comprising:
automatically extracting, by a computer processor, a key phrase from the provided content using a combination of:
a first model associated with a semantic relevance between the key phrase and the provided content, and
a second model associated with generating syntactic boundaries of key phrases in the provided content; and
generating, by the computer processor, a question in natural language using the extracted key phrase as an answer to the question through one or more iterations of interpolating a sequence of words in the question, wherein the interpolation of the sequence of the words is based on a second probability that a word is next in the sequence of words in the question.

22 . The method of claim 21 , wherein the first model is based at least upon a first probability of an entity in the provided content being an answer conditioned on the provided content, wherein a second model is a sequence-to-sequence model for generating one or more boundaries of key phrases, and wherein the entity comprises a word in the provided content.

23 . The method of claim 22 , the method further comprising conditioning a question generation model based on the one or more key phrases, wherein the question generation model is used for generating the question.

24 . The method of claim 21 , the method further comprising:
receiving a set of questions and answers using cloud sourcing, and conditioning the first model for training using the received set of questions and answers.

25 . The method of claim 24 , wherein the neural model has been trained on a dataset comprising human-selected key phrases.

26 . The method of claim 21 , further comprising identifying a start location and an end location of the key phrases in the provided content.

27 . The method of claim 26 , wherein the identifying includes performing a dot product attention mechanism parameterizing a probability distribution.

28 . The method of claim 21 , further comprising determining an attention distribution of word positions in the provided content, wherein the generating includes providing at least one word of the question based on the attention distribution.

29 . The method of claim 21 , wherein the provided content includes a document.

30 . A system for generating questions based on provided content, comprising:
a pointer network to automatically extract, using a machine learning model, a key phrase from the provided content using a combination of:
a first model associated with a semantic relevance between the key phrase and the provided content, and
a second model associated with generating syntactic boundaries of key phrases in the provided content; and
a question generator to generate a question in natural language using the extracted key phrase as an answer to the question through one or more iterations of interpolating a sequence of words in the question, wherein the interpolation of the sequence of the words is based on a second probability that a word is next in the sequence of words in the question.

31 . The system of claim 30 , wherein the first model is based at least upon a first probability of an entity in the provided content being an answer conditioned on the provided content, wherein a second model is a sequence-to-sequence model for generating one or more boundaries of key phrases, wherein the entity comprises a word in the provided content, and wherein the pointer network identifies start and end locations of the key phrases in the provided content.

32 . The system of claim 30 , wherein the pointer network comprises:
an encoder for encoding the provided content; and a decoder to extract the key phrases.

33 . The system of claim 30 , wherein the question generator comprises:
an encoder for encoding the provided content; and a decoder to generate the question.

34 . The system of claim 33 , wherein the decoder includes an attention mechanism.

35 . The system of claim 33 , wherein the decoder includes a Long Short Term Memory (LSTM).

36 . The system of claim 30 , wherein the pointer network receives a set of questions and answers using cloud sourcing and conditions the first model for training using the received set of questions and answers.

37 . A storage device storing a program having instructions which, when executed by a computer processor, cause the processor to execute a method for generating questions based on provided content, comprising:
automatically extracting, by a computer processor, a key phrase from the provided content using a combination of:
a first model associated with a semantic relevance between the key phrase and the provided content, and
a second model associated with generating syntactic boundaries of key phrases in the provided content; and
generating, by the computer processor, a question in natural language using the extracted key phrase as an answer to the question through one or more iterations of interpolating a sequence of words in the question, wherein the interpolation of the sequence of the words is based on a second probability that a word is next in the sequence of words in the question.

38 . The storage device of claim 37 , wherein the first model is based at least upon a first probability of an entity in the provided content being an answer conditioned on the provided content, wherein a second model is a sequence-to-sequence model for generating one or more boundaries of key phrases, and wherein the entity comprises a word in the provided content.

39 . The storage device of claim 37 , wherein the method further comprises conditioning a question generation model based on the one or more key phrases, wherein the question generation model is used for generating the question.

40 . The storage device of claim 37 , the method further comprising:
receiving a set of questions and answers using cloud sourcing; and conditioning the first model for training using the received set of questions and answers.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Yuan Xingdi
• Wang Tong
• Trischler Adam Peter
• Subramanian Sandeep

Classifications

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/09

  Supervised learning · CPC title