Learning with limited supervision for question-answering with light-weight Markov models

Patent metadata

Field	Value
Publication number	US-12061995-B2
Application number	US-202016813098-A
Country	US
Kind code	B2
Filing date	Mar 9, 2020
Priority date	Mar 9, 2020
Publication date	Aug 13, 2024
Grant date	Aug 13, 2024

Abstract

Methods for natural language semantic matching performed by training and using a Markov Network model are provided. The trained Markov Network model can be used to identify answers to questions. Training may be performed using question-answer pairs that include labels indicating a correct or incorrect answer to a question. The trained Markov Network model can be used to identify answers to questions from sources stored on a database. The Markov Network model provides superior performance over other semantic matching models, in particular, where the training data set includes a different information domain type relative to the input question or the output answer of the trained Markov Network model.

First claim

What is claimed is:

1. One or more computer storage media having computer-useable instructions that, when used by one or more computing devices, cause the one or more computing devices to train a Markov Network model for identifying answers to questions by performing operations comprising:
receiving training data comprising question-answer pairs;
identifying object texts and relation texts within the question-answer pairs;
generating a Markov Network comprising a node network of object nodes and relation nodes, the Markov Network connecting object nodes of questions with object nodes of answers and connecting relation nodes of questions with relation nodes of answers from the question-answer pairs, wherein the object nodes are based on the identified object texts and the relation nodes are based on the identified relation texts;
training the Markov Network by optimizing parameters of feature functions defined on the object nodes and relation nodes, the training resulting in a trained Markov Network model; and
storing the trained Markov Network model for use in identifying an answer to a question.

2. The media of claim 1 , wherein the question-answer pairs are associated with binary labels, and wherein the binary labels comprise a first label that indicates a correct answer to a question and a second label that indicates an incorrect answer to the question.

3. The media of claim 1 , wherein the training data comprises question-answer pairs associated with a non-specific domain type and the trained Markov Network model is for use in identifying an answer from a specific domain type, the non-specific domain type comprising multiple information categories and the specific domain type comprising a specific information category.

4. The media of claim 1 , wherein generating the Markov Network further comprises:
forming a relation side structure of the Markov Network that includes the relation nodes, each relation node representing an identified relation text, the relation side structure comprising a plurality of relation binary cliques that each include a relation node pair having a relation first node of a first relation text identified from a question and a relation second node of a second relation text identified from an answer of a question-answer pair, wherein each relation first node for each relation node pair of the plurality of relation binary cliques is dependent upon each relation second node for each relation node pair of the plurality of relation node pairs; and
forming an object side structure of the Markov Network that includes the object nodes, each object node representing an identified object text, the object side structure comprising a plurality of object binary cliques that each include an object node pair having an object first node of a first object text identified from the question and an object second node of a second object text identified from the answer of the question-answer pair, wherein each object first node for each object node pair of the plurality of object binary cliques is dependent upon each object second node for each object node pair of the plurality of object node pairs.

5. The media of claim 4 , wherein:
a relative position of each of the relation first nodes is determined from a relative position of each of the identified relation texts within the question-answer pair, and wherein each of the relation first nodes is dependent upon adjacent relation first nodes; and
a relative position of each of the object first nodes is determined from a relative position of each of the identified object texts within the question-answer pair, and wherein each of the object first nodes is dependent upon adjacent object first nodes.

6. The media of claim 5 , wherein an exponential decay factor is applied to the Markov Network based on the relative position of each of the relation first nodes and the relative position of each of the object first nodes.

7. The media of claim 4 , wherein each feature function generates a probability score indicating a similarity between each of the plurality of relation binary cliques and each of the plurality of object binary cliques.

8. The media of claim 7 , wherein training the Markov Network further comprises jointly optimizing the parameters using a maximum likelihood as optimization criteria.

9. The media of claim 1 , wherein each feature function is selected from a deep learning model or an information retrieval model.

10. The media of claim 8 , wherein the feature functions are Bidirectional Encoder Representations from Transformers (BERT).

11. A computerized method using a trained Markov Network model for identifying answers to questions, the method comprising:
receiving a question from a user computing device;
identifying an answer to the question using a trained Markov Network model, the trained Markov Network model determined by training a Markov Network comprising a node network of object nodes and relation nodes, the Markov Network connecting object nodes of questions with object nodes of answers and connecting relation nodes of questions with relation nodes of answers from question-answer pairs, the object nodes based on object texts and the relation nodes based on relation texts identified from the question-answer pairs of training data; and
providing the answer to the user computing device in response to receiving the question.

12. The method of claim 11 , wherein the question received from the computing device is associated with a specific domain type and the training data is associated with a non-specific domain type, the non-specific domain type comprising multiple information categories and the specific domain type comprising a specific information category.

13. The method of claim 11 , wherein the training data comprises a first portion associated with a non-specific domain type and a second portion associated with a specific domain type, and the answer is identified from a dataset of the specific domain type, the non-specific domain type comprising multiple information categories and the specific domain type comprising a specific information category.

14. The method of claim 11 , wherein the Markov Network comprises:
a relation side structure that includes the relation nodes, each relation node representing an identified relation text, the relation side structure comprising a plurality of relation binary cliques that each include a relation node pair having a relation first node of a first relation text identified from a question and a relation second node of a second relation text identified from an answer of a question-answer pair, wherein each relation first node for each relation node pair of the plurality of relation binary cliques is dependent upon each relation second node for each relation node pair of the plurality of relation node pairs; and
an object side structure that includes the object nodes, each object node representing an identified object text, the object side structure comprising a plurality of object binary cliques that each include an object node pair having an object first node of a first object text identified from the question and an object second node of a second object text identified from the answer of the question-answer pair, wherein each object first node for each object node pair of the plurality of object binary cliques is dependent upon each object second node for each object node pair of the plurality of object node pairs.

15. The method of claim 14 , wherein:
each of the relation first nodes is dependent upon adjacent relation first nodes; and
each of the object first nodes is dependent upon adjacent objec

Assignees

• Adobe Inc

Inventors

• Bui Trung Huu
• Sun Tong
• Modani Natwar
• Wang Lidan
• Dernoncourt Franck

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06F40/205

  Parsing · CPC title

• G06F40/279

  Recognition of textual entities · CPC title

• G06F40/30 Primary

  Semantic analysis · CPC title

• G06N20/00

  Machine learning · CPC title