Method and apparatus for classifying class, to which sentence belongs, using deep neural network

Patent metadata

Field	Value
Publication number	US-11568240-B2
Application number	US-201816613317-A
Country	US
Kind code	B2
Filing date	May 16, 2018
Priority date	May 16, 2017
Publication date	Jan 31, 2023
Grant date	Jan 31, 2023

Abstract

Provided are a method and apparatus for classifying a sentence into a class by using a deep neural network. The method includes respectively training first and second sentences by using first and second neural networks, obtaining a contrastive loss based on first and second feature vectors generated as output data of the training, and information about whether classes to which the first and second sentences belong are the same, and repeating the training in such a manner that the contrastive loss has a maximum value.

First claim

The invention claimed is:

1. A method of classifying a sentence into a class by using a deep neural network, the method comprising:
training a first feature vector by using a first neural network and using a first sentence comprising one or more words, and a first class to which the first sentence belongs, as input data;
training a second feature vector by using a second neural network and using a second sentence and a second class to which the second sentence belongs, as input data;
obtaining a contrastive loss quantifying a representational similarity between the first and second sentences by using a dot product of the first feature vector and the second feature vector, and an equation for indicating, as a number, information about whether the first class is the same as the second class; and
repeating the trainings using the first and second neural networks, in such a manner that the contrastive loss has a maximum value.

2. The method of claim 1 , further comprising:
receiving an utterance input from a user;
recognizing the received utterance as a sentence; and
extracting one or more words comprised in the recognized sentence, and transforming the one or more words into one or more word vectors,
wherein the training of the first feature vector comprises:
generating a sentence vector by arranging the one or more word vectors in a form of a matrix, and
training the first feature vector by inputting the sentence vector to the first neural network as input data.

3. The method of claim 1 ,
wherein a plurality of sentences and a plurality of classes to which the plurality of sentences belong are stored in a database, and
wherein the second sentence and the second class are randomly extracted from the database.

4. The method of claim 1 , wherein the equation outputs 1 when the first class is the same as the second class, and outputs 0 when the first class is not the same as the second class.

5. The method of claim 1 , wherein the training using the first neural network and the training using the second neural network are simultaneously performed.

6. An electronic device for classifying a sentence into a class by using a deep neural network, the electronic device comprising
a processor configured to perform training by using a neural network,
wherein the processor is further configured to:
train a first feature vector by using a first neural network and using a first sentence comprising one or more words, and a first class to which the first sentence belongs, as input data,
train a second feature vector by using a second neural network and using a second sentence and a second class to which the second sentence belongs, as input data,
obtain a contrastive loss quantifying a representational similarity between the first and second sentences by using a dot product of the first feature vector and the second feature vector, and an equation for indicating, as a number, information about whether the first class is the same as the second class, and

repeat the trainings using the first and second neural networks, in such a manner that the contrastive loss has a maximum value.

7. The electronic device of claim 6 , further comprising an utterance inputter configured to receive an utterance input from a user,
wherein the processor is further configured to recognize the received utterance as a sentence, and extract one or more words comprised in the recognized sentence, and transform the one or more words into one or more word vectors.

8. The electronic device of claim 7 , wherein the processor is further configured to generate a sentence vector by arranging the one or more word vectors in a form of a matrix, and train the first feature vector by inputting the sentence vector to the first neural network as input data.

9. The electronic device of claim 6 , further comprising:
a database storing a plurality of sentences and a plurality of classes to which the plurality of sentences belong,
wherein the processor is further configured to randomly extract the second sentence and the second class from the database and input the second sentence and the second class to the second neural network as input data.

10. The electronic device of claim 6 , wherein the equation outputs 1 when the first class is the same as the second class, and outputs 0 when the first class is not the same as the second class.

11. The electronic device of claim 6 , wherein the processor is further configured to transform the first sentence into a matrix comprising one or more word vectors, input the transformed matrix to a convolutional neural network (CNN) as input data, generate feature maps by applying a plurality of filters, and extract the first feature vector by passing the feature maps through a max pooling layer.

12. The electronic device of claim 6 , wherein the processor is further configured to simultaneously perform the training using the first neural network and the training using the second neural network.

13. A computer program product comprising a non-transitory computer-readable recording medium,
the non-transitory computer-readable recording medium comprising instructions that are executed to:
train a first feature vector by using a first neural network and using a first sentence comprising one or more words, and a first class to which the first sentence belongs, as input data;
train a second feature vector by using a second neural network and using a second sentence and a second class to which the second sentence belongs, as input data,
obtain a contrastive loss quantifying a representational similarity between the first and second sentences by using a dot product of the first feature vector and the second feature vector, and an equation for indicating, as a number, information about whether the first class is the same as the second class, and
repeat the trainings using the first and second neural networks, in such a manner that the contrastive loss has a maximum value.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Song Hee-Jun
• Kulkarni Nilesh

Classifications

• G06F40/30 Primary

  Semantic analysis · CPC title

• G06N3/08 Primary

  Learning methods · CPC title

• G06N3/0454

  Physics · mapped topic

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title