Systems, methods, and apparatuses for controlling output length in neural machine translation

Patent metadata

Field	Value
Publication number	US-11295081-B1
Application number	US-201916586293-A
Country	US
Kind code	B1
Filing date	Sep 27, 2019
Priority date	Sep 27, 2019
Publication date	Apr 5, 2022
Grant date	Apr 5, 2022

Abstract

Techniques for neural machine translation with a controlled output are described. An exemplary method includes receiving a request to perform a machine language translation of text using a translation model; determining a desired target length of the text; using the translation model to translate the text, the identified translation model including an encoder and decoder portion, the decoder portion in accept as an input into a decoder stack at least an embedding of a token of the text, a position of the token within the text, and an indication of length; and output a result of the machine language translation to a requester.

First claim

What is claimed is:

1. A computer-implemented method comprising:
receiving a request from an edge device to perform a machine language translation of text supplied by the edge device using an identified translation model;
determining a desired target length of the text;
using the identified translation model to translate the text, the identified translation model including an encoder and decoder portion, the decoder portion accepts as an input into a decoder stack an embedding of a token of the text, a position of the token within the text, and a relative position of the token with respect to the desired target length of text; and
outputting a result of the machine language translation to a requester.

2. The computer-implemented method of claim 1 , wherein the relative position is based on a number of characters used per token and the desired target length based on a number of characters.

3. The computer-implemented method of claim 1 , wherein the identified translation model comprises a plurality of identical encoder layers composed of an attention-based sub-layer followed by a first position-wise feedforward network, and a plurality of identical decoder layers composed of at least two attention-based sub-layers followed by a second position-wise feedforward network.

4. A computer-implemented method comprising:
receiving a request to perform a machine language translation of text using a translation model;
determining a desired target length of the text;
using the translation model to translate the text, the translation model including an encoder and decoder portion, the decoder portion to accept as an input into a decoder stack at least an embedding of a token of the text, a position of the token within the text, and an indication of length; and
outputting a result of the machine language translation to a requester.

5. The computer-implemented method of claim 4 , wherein the indication of length is a relative length encoding of a relative position of a given token with respect to the desired target length.

6. The computer-implemented method of claim 4 , wherein the indication of length is an absolute length encoding of an absolute position of a given token with respect to the desired target length.

7. The computer-implemented method of claim 4 , wherein the translation model has been trained using a different length token for a plurality of groups of target/source ratios and the indication of length is a length token for a particular group of the plurality of groups of target/source ratios.

8. The computer-implemented method of claim 4 , wherein the indication of length is one of a combination of two of relative length encoding, an absolute length encoding, and a length token for a particular target/source ratio.

9. The computer-implemented method of claim 4 , wherein the text is closed captioning data extracted from an audio/video file.

10. The computer-implemented method of claim 4 , wherein the translation model comprises a plurality of identical encoder layers composed of an attention-based sub-layer followed by a first position-wise feedforward network, and a plurality of identical decoder layers composed of at least two attention-based sub-layers followed by a second position-wise feedforward network.

11. The computer-implemented method of claim 4 , wherein the text is generated by performing automatic speech recognition on audio data.

12. The computer-implemented method of claim 4 , wherein the translation model is specific for a conversion from a particular source language to a particular target language and the request includes an identifier of the translation model.

13. The computer-implemented method of claim 4 , wherein the method is performed on an edge device.

14. The computer-implemented method of claim 4 , further comprising:
performing an action in response to the outputted result of the machine language translation.

15. A system comprising:
an edge device to provide text to be translated; and
a neural machine translation service implemented by a second one or more electronic devices, the neural machine translation service including instructions that upon execution cause the neural machine translation service to:
receive a request to perform a machine language translation of the provided text using a translation model;
determine a desired target length of the text;
using the translation model to translate the text, the translation model including an encoder and decoder portion, the decoder portion to accept as an input into a decoder stack at least an embedding of a token of the text, a position of the token within the text, and an indication of length; and
output a result of the machine language translation to a requester.

16. The system of claim 15 , wherein the indication of length is a relative length encoding of a relative position of a given token with respect to the desired target length.

17. The system of claim 15 , wherein the indication of length is an absolute length encoding of an absolute position of a given token with respect to the desired target length.

18. The system of claim 15 , wherein the translation model has been trained using a different length token for a plurality of groups of target/source ratios and the indication of length is a length token for a particular group of the plurality of groups of target/source ratios.

19. The system of claim 15 , wherein the indication of length is one of a combination of two of relative length encoding, an absolute length encoding, and a length token for a particular target/source ratio.

20. The system of claim 15 , wherein the translation model comprises a plurality of identical encoder layers composed of an attention-based sub-layer followed by a first position-wise feedforward network, and a plurality of identical decoder layers composed of at least two attention-based sub-layers followed by a second position-wise feedforward network.

Assignees

• Amazon Tech Inc

Inventors

• Federico Marcello
• Di Gangi Mattia Antonino
• Lakew Surafel Melaku

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/0499

  Feedforward networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/08

  Learning methods · CPC title