Automatic post-editing model for generated natural language text

Patent metadata

Field	Value
Publication number	US-12039286-B2
Application number	US-202217700123-A
Country	US
Kind code	B2
Filing date	Mar 21, 2022
Priority date	Jul 15, 2019
Publication date	Jul 16, 2024
Grant date	Jul 16, 2024

Abstract

Techniques are disclosed for training and/or utilizing an automatic post-editing model in correcting translation error(s) introduced by a neural machine translation model. The automatic post-editing model can be trained using automatically generated training instances. A training instance is automatically generated by processing text in a first language using a neural machine translation model to generate text in a second language. The text in the second language is processed using a neural machine translation model to generate training text in the first language. A training instance can include the text in the first language as well as the training text in the first language.

First claim

What is claimed is:

1. A method implemented by one or more processors, the method comprising:
detecting a spoken utterance at a client device, wherein the spoken utterance includes an indication of one or more device actions of a device associated with the client device, and wherein the spoken utterance is directed to an automated assistant executing at least in part on the client device;
in response to the spoken utterance being detected at the client device:
generating, by the automated assistant, natural language text, wherein generating the natural language text comprises applying, to a text template, structured data that is responsive to the spoken utterance;
processing the natural language text generated based on applying the structured data that is responsive to the spoken utterance to the text template, using an automatic post-editing model to generate edited text, wherein the edited text, generated based on processing the text using the automatic post-editing model, corrects one or more errors in the natural language text; and
causing the client device to perform one or more actions based on the edited text, wherein causing the client device to perform the one or more actions based on the edited text comprises:
processing the edited text to determine the one or more device actions of the device associated with the client device; and
causing the device to perform the one or more device actions.

2. The method of claim 1 , wherein the one or more errors in the natural language text include a subject-verb agreement error in the natural language text.

3. The method of claim 1 , wherein the text template comprises one or more fixed terms and one or more variables, and wherein generating the natural language text comprises using the set of structured data in populating the one or more variables.

4. The method of claim 1 , wherein the automatic post-editing model is a transformer model that includes a transformer encoder and a transformer decoder, and wherein processing the natural language text using the automatic post-editing model to generate the edited text comprises:
processing the natural language text using the transformer encoder to generate an encoded representation of the natural language text; and
generating the edited text, token-by-token, using the transformer decoder attended to the encoded representation of the natural language text.

5. The method of claim 4 , wherein generating the predicted output, token-by-token, using the transformer decoder attended to the encoded representation of the natural language text comprises:
generating, at each of a plurality of iterations of processing using the transformer decoder, a probability distribution over a vocabulary of tokens;
selecting, from the vocabulary of tokens and based on the probability distribution for the iteration, a corresponding token for the iteration; and
using the selected token as part of input to the transformer decoder in a subsequent iteration of the iterations of processing.

6. The method of claim 1 , wherein the automatic post-editing model is a sequence to sequence model.

7. The method of claim 1 , wherein causing the client device to perform one or more actions based on the edited text comprises:
processing the edited text using a text to speech engine to generate an audio waveform corresponding to the edited text; and
causing the client device to render the audio waveform via one or more speakers of the client device.

8. The method of claim 1 , wherein the device associated with the client device is a light, a thermostat, or a camera.

9. The method of claim 1 , wherein the automatic post-editing model is trained using a training instance including training text and ground truth text, and wherein the training text is generated by:
processing the ground truth text using random noise to generate the training text wherein processing the ground truth text using random noise to generate the training text comprises inserting one or more words into the ground truth text, deleting one or more words from the ground truth text, and/or reordering one or more words in the ground truth text.

10. The method of claim 9 , wherein processing the ground truth text using random noise to generate the training text comprises:
inserting one or more words into the ground truth text.

11. The method of claim 10 , wherein processing the ground truth text using random noise to generate the training text comprises:
reordering one or more words in the ground truth training text.

12. The method of claim 9 , wherein processing the ground truth text using random noise to generate the training text comprises:
deleting one or more words from the ground truth text.

13. The method of claim 1 , wherein the automatic post-editing model is trained for use in correcting one or more translation errors introduced by a neural machine translation model translating text from a source language into a target language.

14. A system, comprising:
memory storing instructions; and
one or more processors executing the instructions to:
detect a spoken utterance at a client device, wherein the spoken utterance includes an indication of one or more device actions of a device associated with the client device, and wherein the spoken utterance is directed to an automated assistant executing at least in part on the client device;
in response to the spoken utterance being detected at the client device:
generate natural language text, wherein in generating the natural language text one or more of the processors are to apply, to a text template, structured data that is responsive to the spoken utterance;
process the natural language text generated based on applying the structured data that is responsive to the spoken utterance to the text template, using an automatic post-editing model to generate edited text, wherein the edited text, generated based on processing the text using the automatic post-editing model, corrects one or more errors in the natural language text; and
cause the client device to perform one or more actions based on the edited text, wherein causing the client device to perform the one or more actions based on the edited text comprises:
process the edited text to determine the one or more device actions of the device associated with the client device; and
cause the device to perform the one or more device actions.

15. The system of claim 14 , wherein the text template comprises one or more fixed terms and one or more variables, and wherein generating the natural language text comprises using the set of structured data in populating the one or more variables.

16. The system of claim 14 , wherein the automatic post-editing model is a transformer model that includes a transformer encoder and a transformer decoder, and wherein in processing the natural language text using the automatic post-editing model to generate the edited text one or more of the processors are to:
process the natural language text using the transformer encoder to generate an encoded representation of the natural language text; and
generate the edited text, token-by-token, using the transformer decoder attended to the encoded representation of the natural language text.

17. The system of claim 14 , wherein in generating the predicted output, token-by-token, using the transformer decoder attended to the encoded representation of the natural language text one or more of the processors are to:
generate, at each of a plurality of iterations of processing using the transformer decoder, a probability distribution over a vocabulary

Assignees

• Google Llc

Inventors

• Freitag Markus
• Caswell Isaac
• Roy Howard Scott

Classifications

• G06F40/253

  Grammatical analysis; Style critique · CPC title

• G06F40/166

  Editing, e.g. inserting or deleting · CPC title

• G10L13/00

  Speech synthesis; Text to speech systems · CPC title

• G06F40/58

  Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation · CPC title

• G06F40/51 Primary

  Translation evaluation · CPC title