Method and apparatus for generating model

Patent metadata

Field	Value
Publication number	US-11501182-B2
Application number	US-201916502701-A
Country	US
Kind code	B2
Filing date	Jul 3, 2019
Priority date	Jul 3, 2018
Publication date	Nov 15, 2022
Grant date	Nov 15, 2022

Abstract

A method and an apparatus for generating a model are provided. The method includes: acquiring a sample set including sample sentences and labeling knowledge corresponding thereto; and selecting a sample from the sample set, and performing following training steps: inputting a sample sentence into a first initial model to generate first prediction knowledge corresponding to the sample sentence; inputting the first prediction knowledge into a second initial model to generate a first prediction sentence corresponding to the first prediction knowledge; inputting labeling knowledge into the second initial model to generate a second prediction sentence corresponding to the labeling knowledge; inputting the second prediction sentence into the first initial model to generate a second prediction knowledge corresponding to the second prediction sentence; determining a first reward signal; and training, using a reinforcement learning method based on the first reward signal to obtain a first model.

First claim

What is claimed is:

1. A method for generating a model, the method comprising:
acquiring a sample set, the sample set comprising sample sentences and labeling knowledge corresponding to the sample sentences; and
selecting a sample from the sample set, and performing following training steps: inputting a sample sentence in the selected sample into a first initial model to generate first prediction knowledge corresponding to the sample sentence; inputting the first prediction knowledge into a second initial model to generate a first prediction sentence corresponding to the first prediction knowledge; inputting labeling knowledge into the second initial model to generate a second prediction sentence corresponding to the labeling knowledge; inputting the second prediction sentence into the first initial model to generate a second prediction knowledge corresponding to the second prediction sentence; determining a first reward signal according to at least one of following information items: a degree of the first prediction knowledge conforming to a preset knowledge expression rule, a similarity between the first prediction knowledge and the labeling knowledge, and a probability that the second prediction knowledge is the labeling knowledge; and training, using a reinforcement learning method based on the determined first reward signal, to obtain a first model.

2. The method according to claim 1 , wherein the training steps further comprise:
determining a second reward signal according to at least one of following information items: a degree of the second prediction sentence conforming to a preset language expression rule, a similarity between the second prediction sentence and the sample sentence, and a probability that the first prediction sentence is the sample sentence; and
training, using the reinforcement learning method based on the determined second reward signal, to obtain a second model.

3. The method according to claim 2 , wherein the first initial model comprises an encoder and a decoder; and
the inputting a sample sentence in the selected sample into a first initial model to generate first prediction knowledge corresponding to the sample sentence, comprises:
constructing an input sequence based on the sample sentence;
mapping the input sequence to an input hidden state sequence using the encoder, and mapping an output sequence to an output hidden state sequence using the decoder;
decoding the input hidden state sequence using the decoder to generate a prediction state sequence; and
obtaining the first prediction knowledge based on the prediction state sequence.

4. The method according to claim 3 , wherein the decoding the input hidden state sequence using the decoder to generate a prediction state sequence, comprises:
acquiring, for a target position in a to-be-generated prediction state sequence, a state of an hidden layer of the decoder after the decoder acquires a prediction state of a last position prior to the target position by decoding, as a current hidden state of the decoder;
calculating matching degrees between input hidden states in the input hidden state sequence and a prediction state of the target position in the to-be-generated prediction state sequence based on the current hidden state;
calculating attention weights of the input hidden states on the prediction state of the target position based on the matching degrees;
performing a weighted sum of the input hidden states according to the attention weights to obtain a context vector;
calculating a probability distribution of the prediction state of the target position based on the context vector, an output hidden state of the last position prior to the target position in the output hidden state sequence, and a state of the hidden layer of the decoder when the hidden layer of the decoder decodes the prediction state of the target position; and
determining the prediction state of the target position based on the probability distribution.

5. The method according to claim 4 , wherein a probability of the prediction state of the target position is: a sum of a probability of copying a target word from a corresponding sample sentence as a target object in the output sequence and a probability of selecting a target symbol from a preset symbol set and using an object represented by the target symbol as an object in the output sequence; and
symbols in the preset symbol set are used in conjunction with words in the sample sentence to fully represent one of following knowledge in the sample sentence: knowledge based on a verb or a preposition, knowledge based on a noun attribute, entity description knowledge and knowledge of a relationship between an entity and a concept.

6. The method according to claim 5 , wherein the method further comprises:
updating, in response to copying a target word from a corresponding sample sentence as an object in the output sequence, a probability of copying the target word from the corresponding sample sentence as the object in the output sequence to zero.

7. An apparatus for generating a model, the apparatus comprising:
at least one processor; and
a memory storing instructions, wherein the instructions, when executed by the at least one processor, cause the at least one processor to perform operations, the operations comprising:
acquiring a sample set, the sample set comprising sample sentences and labeling knowledge corresponding to the sample sentences; and
selecting a sample from the sample set, and perform following training steps: inputting a sample sentence in the selected sample into a first initial model to generate first prediction knowledge corresponding to the sample sentence; inputting the first prediction knowledge into a second initial model to generate a first prediction sentence corresponding to the first prediction knowledge; inputting labeling knowledge into the second initial model to generate a second prediction sentence corresponding to the labeling knowledge; inputting the second prediction sentence into the first initial model to generate a second prediction knowledge corresponding to the second prediction sentence; determining a first reward signal according to at least one of following information items: a degree of the first prediction knowledge conforming to a preset knowledge expression rule, a similarity between the first prediction knowledge and the labeling knowledge, and a probability that the second prediction knowledge is the labeling knowledge; and training, using a reinforcement learning method based on the determined first reward signal to obtain a first model.

8. The apparatus according to claim 7 , wherein the training steps further comprise:
determining a second reward signal according to at least one of following information items: a degree of the second prediction sentence conforming to a preset language expression rule, a similarity between the second prediction sentence and the sample sentence, and a probability that the first prediction sentence is the sample sentence; and
training, using the reinforcement learning method based on the determined second reward signal to obtain a second model.

9. The apparatus according to claim 8 , wherein the first initial model comprises an encoder and a decoder; and
the inputting a sample sentence in the selected sample into a first initial model to generate first prediction knowledge corresponding to the sample sentence, comprises:
constructing an input sequence based on the sample sentence;
mapping the input sequence to an input hidden state sequence using the encoder, and mapping an output sequence to an output hidden state sequence using the decoder;
decoding the input hidden state sequence using the decoder to generate a prediction sta

Assignees

• Baidu online network technology beijing co ltd

Inventors

• Sun Mingming
• Li Xu
• Li Ping

Classifications

• G06F40/30 Primary

  Semantic analysis · CPC title

• G06N5/025 Primary

  Extracting rules from data · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N3/045

  Combinations of networks · CPC title