Training and operating multi-layer computational models

Patent metadata

Field	Value
Publication number	US-10445650-B2
Application number	US-201514949156-A
Country	US
Kind code	B2
Filing date	Nov 23, 2015
Priority date	Nov 23, 2015
Publication date	Oct 15, 2019
Grant date	Oct 15, 2019

Abstract

A processing unit can successively operate layers of a multilayer computational graph (MCG) according to a forward computational order to determine a topic value associated with a document based at least in part on content values associated with the document. The processing unit can successively determine, according to a reverse computational order, layer-specific deviation values associated with the layers based at least in part on the topic value, the content values, and a characteristic value associated with the document. The processing unit can determine a model adjustment value based at least in part on the layer-specific deviation values. The processing unit can modify at least one parameter associated with the MCG based at least in part on the model adjustment value. The MCG can be operated to provide a result characteristic value associated with test content values of a test document.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
successively operating layers of a multilayer computational graph according to a forward computational order to determine a topic value associated with a first document of a plurality of documents based at least in part on content values associated with the first document;
successively determining, according to a reverse computational order, layer-specific deviation values associated with individual layers of the multilayer computational graph based at least in part on the topic value, the content values, and a characteristic value associated with the first document;
determining a model adjustment value based at least in part on the layer-specific deviation values;
modifying at least one parameter associated with the multilayer computational graph based at least in part on the model adjustment value;
determining a first candidate characteristic value and a second candidate characteristic value;
determining, for individual content values of the content values, a respective first result likelihood associated with the first candidate characteristic value and a respective second result likelihood associated with the second candidate characteristic value; and
ranking the individual content values of the content values based at least in part on respective comparisons of the respective first result likelihoods with the respective second result likelihoods.

2. A computer-implemented method as recited in claim 1 , further comprising operating the multilayer computational graph to provide a result characteristic value associated with a test document based at least in part on test content values associated with the test document.

3. A computer-implemented method as recited in claim 1 , further comprising successively operating the layers of the multilayer computational graph according to the first computational order to determine a second topic value associated with a second document of the plurality of documents based at least in part on second content values associated with the second document;
successively determining, according to the reverse computational order, second layer-specific deviation values associated with individual layers of the multilayer computational graph based at least in part on the second topic value, second content values associated with the second document, and a second characteristic value associated with the second document; and
determining the model adjustment value further based at least in part on the second layer-specific deviation values.

4. A computer-implemented method as recited in claim 3 , further comprising:
performing, with respect to multiple groups of a first document and a second document, the determining the topic values, the determining the layer-specific deviation values, the determining the second layer-specific deviation values, the determining the model adjustment value based at least in part on the layer-specific deviation values and the second layer-specific deviation values, and the modifying.

5. A computer-implemented method as recited in claim 1 , wherein the documents comprise named fields and the individual ones of the content values comprise values of the named fields.

6. A computer-implemented method as recited in claim 1 , wherein the documents comprise free-form user text and the individual ones of the content values comprise words or phrases in the free-form user text.

7. A computer-implemented method as recited in claim 1 , wherein the ranking comprises: determining the respective comparisons as respective log-likelihood ratios of the respective first result likelihoods and the respective second result likelihoods.

8. A system, comprising:
one or more computer-readable media having stored thereon a plurality of modules; and
one or more processing units operably coupled to at least one of the computer-readable media, the one or more processing units adapted to execute modules of the plurality of modules comprising:
an estimation engine configured to successively operate layers of a multilayer computational graph according to a forward computational order to determine topic values of a document based at least in part on content values associated with the document;
a classification engine configured to determine one or more characteristic values associated with the document based at least in part on the topic values, wherein:
the estimation engine is further configured to operate, for individual documents of one or more documents, the layers of the multilayer computational graph according to the forward computational order to determine a set of topic values associated with each individual document based at least in part on a set of content values of that document; and
the classification engine is further configured to determine, for individual documents of the one or more documents, a set of characteristic values of each individual document based at least in part on the set of topic values associated with that document; and

a ranking engine configured to:
determine a first candidate characteristic value and a second candidate characteristic value;
determine candidate content values of the content values associated with the documents of the one or more documents;
determine, for individual candidate content values, a respective first result likelihood associated with the first candidate characteristic value and a respective second result likelihood associated with the second candidate characteristic value; and
rank the individual candidate content values based at least in part on respective comparisons of the respective first result likelihoods with the respective second result likelihoods.

9. A system as recited in claim 8 , wherein the classification engine is configured to determine the one or more characteristic values associated with the document by:
determining probabilities of respective candidate characteristic values based at least in part on the topic values; and
selecting, as the one or more characteristic values, one or more of the candidate characteristic values having respective probabilities meeting one or more selected criteria.

10. A system as recited in claim 9 , wherein the one or more selected criteria comprise at least one of:
a requirement that the respective probability exceed a selected threshold,
a requirement that the respective probability be the highest probability of the determined probabilities, or
a requirement that the respective probability be one of the N highest probabilities of the determined probabilities, wherein N is a positive integer.

11. A system as recited in claim 8 , wherein the documents comprise named fields and the individual candidate content values comprise values of the named fields.

12. A system as recited in claim 8 , wherein the documents comprise free-form user text and the individual candidate content values comprise words or phrases in the free-form user text.

13. A system as recited in claim 8 , wherein the multilayer computational graph comprises a fixed number of the layers.

14. A computer-readable medium having thereon computer-executable instructions, the computer-executable instructions upon execution to configure a computer to perform operations comprising:
successively operating layers of a multilayer computational graph according to a first computational order to determine a topic value associated with a document based at least in part on content values associated with the document;
successively determining, according to a second, different computational order, layer-specific deviation values associated with individual layers of th

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Gao Jianfeng
• Deng Li
• He Xiaodong
• Xiao Lin
• Song Xinying
• Shen Yelong
• He Ji
• Chen Jianshu

Classifications

• G06N7/01 Primary

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06N7/005 Primary

  Physics · mapped topic