Deep neural network-based decision network

Patent metadata

Field	Value
Publication number	US-2022164635-A1
Application number	US-202217670368-A
Country	US
Kind code	A1
Filing date	Feb 11, 2022
Priority date	Mar 15, 2017
Publication date	May 26, 2022
Grant date	—

Abstract

The technology disclosed proposes using a combination of computationally cheap, less-accurate bag of words (BoW) model and computationally expensive, more-accurate long short-term memory (LSTM) model to perform natural processing tasks such as sentiment analysis. The use of cheap, less-accurate BoW model is referred to herein as “skimming”. The use of expensive, more-accurate LSTM model is referred to herein as “reading”. The technology disclosed presents a probability-based guider (PBG). PBG combines the use of BoW model and the LSTM model. PBG uses a probability thresholding strategy to determine, based on the results of the BoW model, whether to invoke the LSTM model for reliably classifying a sentence as positive or negative. The technology disclosed also presents a deep neural network-based decision network (DDN) that is trained to learn the relationship between the BoW model and the LSTM model and to invoke only one of the two models.

First claim

What is claimed is:

1 . A system for performing a selection task using a decision neural network-based classifier, comprising:
a memory storing a plurality of processor-executable instructions for training and operating the decision neural network-based classifier; and a processor that reads the plurality of processor-executable instructions from the memory to perform operations comprising:
constructing a decision training set for the decision neural network-based classifier by:
identifying, based on a confusion matrix generated from performance of a non-recurrent network-based classifier and a recurrent neural network-based classifier, a first subset of inputs inferred by the recurrent neural network-based classifier and a second subset of inputs comprising inputs not in the first subset, and
including, in the decision training set, the first subset of inputs labeled with a first model class label identifying the recurrent neural network-based classifier and the second subset of inputs labeled with a second model class label identifying the non-recurrent neural network-based classifier;

generating, by the decision neural network based classifier, a plurality of selection outputs in response to inputs from the decision training set;
computing a training objective by comparing the plurality of selection outputs with labels from the first subset and the second subset; and
updating the decision neural network based classifier based on the training objective.

2 . The system of claim 1 , wherein the operations further comprise:
at an inference stage, selecting, by the decision neural network-based classifier, on an input-by-input basis, between the non-recurrent neural network-based classifier and the recurrent neural network-based classifier to perform a machine classification task; and
based on output probabilities of the trained decision neural network-based classifier, performing the machine classification task on the given input using either the trained recurrent neural network-based classifier or the trained non-recurrent neural network-based classifier.

3 . The system of claim 2 , wherein the operation further comprise:
selecting the trained recurrent neural network-based classifier when an output probability associated with the first model class label is higher than that of the second model class label; or selecting the trained non-recurrent neural network-based classifier when an output probability associated with the second model class label is higher than that of the first model class label.

4 . The system of claim 1 , wherein the operations further comprise:
training the non-recurrent and recurrent neural network-based classifiers to perform the machine classification task using a training set, the training set comprising training inputs annotated with task class labels defined for the machine classification task; using the trained non-recurrent and recurrent neural network-based classifiers to perform the machine classification task on a validation set, the validation set comprising validation inputs annotated with the task class labels; and training the decision neural network-based classifier using the decision training set to output probabilities for the first and second model class labels on an input-by-input basis, the output probabilities specifying respective likelihoods of selecting the trained recurrent neural network-based classifier and the trained non-recurrent neural network-based classifier.

5 . The system of claim 1 , wherein the non-recurrent network-based classifier and the recurrent neural network-based classifier are separately trained on the decision training set.

6 . The system of claim 1 , wherein the decision training set comprises a plurality of sentences annotated with a sentiment label.

7 . The system of claim 1 , wherein the confusion matrix identifies at least one of:
validation inputs accurately classified by both the trained recurrent neural network-based classifier and the trained non-recurrent neural network-based classifier; validation inputs inaccurately classified by both the trained recurrent neural network-based classifier and the trained non-recurrent neural network-based classifier; validation inputs accurately classified by the trained non-recurrent neural network-based classifier but inaccurately classified by the trained recurrent neural network-based classifier; and validation inputs accurately classified by the trained recurrent neural network-based classifier but inaccurately classified by the trained non-recurrent neural network-based classifier.

8 . The system of claim 1 , wherein the first subset includes validation inputs accurately classified by the trained recurrent neural network-based classifier but inaccurately classified by the trained non-recurrent neural network-based classifier.

9 . The system of claim 1 , wherein the second subset includes at least one of:
validation inputs accurately classified by both the trained recurrent neural network-based classifier and the trained non-recurrent neural network-based classifier; validation inputs inaccurately classified by both the trained recurrent neural network-based classifier and the trained non-recurrent neural network-based classifier; and validation inputs accurately classified by the trained non-recurrent neural network-based classifier but inaccurately classified by the trained recurrent neural network-based classifier.

10 . The system of claim 1 , wherein an operation of updating the decision neural network based classifier comprises:
backpropagating gradients only for fully-connected layers and a new classification layer while keeping weights of the trained non-recurrent neural network-based classifier fixed.

11 . A method for performing a selection task using a decision neural network-based classifier, the method comprising:
constructing a decision training set for the decision neural network-based classifier by:
identifying, based on a confusion matrix generated from performance of a non-recurrent network-based classifier and a recurrent neural network-based classifier, a first subset of inputs inferred by the recurrent neural network-based classifier and a second subset of inputs comprising inputs not in the first subset, and
including, in the decision training set, the first subset of inputs labeled with a first model class label identifying the recurrent neural network-based classifier and the second subset of inputs labeled with a second model class label identifying the non-recurrent neural network-based classifier;
generating, by the decision neural network based classifier, a plurality of selection outputs in response to inputs from the decision training set; computing a training objective by comparing the plurality of selection outputs with labels from the first subset and the second subset; and updating the decision neural network based classifier based on the training objective.

12 . The method of claim 11 , further comprising:
at an inference stage, selecting, by the decision neural network-based classifier, on an input-by-input basis, between the non-recurrent neural network-based classifier and the recurrent neural network-based classifier to perform a machine classification task; and
based on output probabilities of the trained decision neural network-based classifier, performing the machine classification task on the given input using either the trained recurrent neural network-based classifier or the trained non-recurrent neural network-based classifier.

13 . The method of claim 12 , further comprising:
selecting the trained recurrent neural network-bas

Assignees

• Salesforce Com Inc

Inventors

• Johansen Alexander Rosenberg
• Mccann Bryan
• Bradbury James
• Socher Richard

Classifications

• G06V10/82

  using neural networks · CPC title

• G06V10/776

  Validation; Performance evaluation · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/048

  Activation functions · CPC title