Machine translation with adapted neural networks

Patent metadata

Field	Value
Publication number	US-11769019-B1
Application number	US-202016953205-A
Country	US
Kind code	B1
Filing date	Nov 19, 2020
Priority date	Nov 19, 2020
Publication date	Sep 26, 2023
Grant date	Sep 26, 2023

Abstract

A translation system receives examples of translations between a first language and a second language. In response to receiving request to translate a source text from the first language to the second language, the system ranks the examples based on the example's applicability to one or more portions of the source text. The system performs additional training of a neural network that was pre-trained to translate from the first language to the second language, where the additional training is based on one or more top-ranking examples. The system translates the source text to the second language using the additionally trained neural network.

First claim

What is claimed is:

1. A system, comprising:
at least one processor; and
at least one memory, the at least one memory comprising instructions that, in response to execution by the at least one processor, cause the system to at least:
receive two or more examples of translation, wherein an example of translation comprises a first phrase in a first language and a second phrase in a second language, the second phrase a translation of the first phrase;
receive a request to translate a source text from the first language to the second language;
rank the two or more examples with respect to a plurality of examples based, at least in part, on a measure of similarity between the first phrase and one or more portions of the source text;
perform additional training of a neural network trained to translate from the first language to the second language, the additional training based at least in part on one or more top-ranking examples of the two or more examples, and at least a portion of an encoder of the neural network is unchanged during the additional training of the neural network; and
translate, based at least in part on the additionally trained neural network, the source text from the first language to the second language.

2. The system of claim 1 , wherein the system, in response to receiving the request to translate the source text, causes allocation of computing capacity, by one or more computing services, for ranking the two or more examples and performing the additional training.

3. The system of claim 1 , wherein the measure of similarity is calculated based at least in part on identification of a sequence of at least one of characters or words in the first phrase that have a corresponding sequence the source text.

4. The system of claim 1 , the at least one memory comprising further instructions that, in response to execution by the at least one processor, cause the system to at least:
calculate a learning rate based, at least in part, on the measure of similarity;
calculate a number of epochs based, at least in part, on the measure of similarity; and
perform the additional training based, at least in part, on the learning rate and the number of epochs.

5. The system of claim 1 , the at least one memory comprising further instructions that, in response to execution by the at least one processor, cause the system to at least:
translate a first portion of the source text using the additionally trained neural network; and
translate a second portion of the source text using a version of the neural network that has not been additionally trained.

6. A method, comprising:
receiving one or more examples of translations between a first language and a second language;
receiving a request to translate a source text from the first language to the second language;
ranking the one or more examples based, at least in part, on applicability of at least the first language of the one or more examples to one or more portions of the source text;
performing additional training of a neural network trained to translate from the first language to the second language, the additional training based at least in part on selecting one or more top-ranking examples of the one or more ranked examples, and at least a portion of an encoder of the neural network is unchanged during the additional training of the neural network; and
translating, based at least in part on the additionally trained neural network, the source text from the first language to the second language.

7. The method of claim 6 , further comprising:
communicating, in response to receiving the request to translate the source text, with one or more services to request allocation of computing capacity for ranking the one or more examples.

8. The method of claim 6 , wherein the one or more examples are loaded from a storage service in response to receiving the request to translate the source text.

9. The method of claim 6 , wherein an example, of the one or more examples, comprises a phrase in the first language and a corresponding translation of the phrase in the second language.

10. The method of claim 6 , further comprising:
generating a score indicative of similarity between a first language portion of one of the one or more examples and a portion of the source text, wherein the ranking of the one or more examples is based, at least in part, on the score.

11. The method of claim 6 , further comprising:
determining, based at least in part on a score indicative of the applicability of the one or more of the one or more examples to the one or more portions of the source text, a learning rate and a number of epochs for the additional training, wherein the additional training is performed based at least in part on the learning rate and the number of epochs.

12. The method of claim 6 , further comprising:
translating a first portion of the source text using the additionally trained neural network; and
translating a second portion of the source text using a second neural network.

13. The method of claim 6 , wherein a first portion of the source text is translated using the additionally trained neural network based, at least in part, on an applicability score of the one or more of the one or more examples being above a threshold amount.

14. The method of claim 6 , wherein a second portion of the source text is translated using a neural network that has not been additionally trained using the one or more examples.

15. A non-transitory computer-readable storage medium comprising instructions that, in response to execution by at least one processor of a computing device, cause the computing device to at least:
receive a request to translate a source text from a first language to a second language;
load, from storage, one or more examples of translations between the first language and the second language;
rank the two or more examples based, at least in part, on applicability of at least the first language of the one or more of the two or more examples to one or more portions of the source text;
perform additional training of a neural network trained to translate from the first language to the second language, the additional training based at least in part on one or more top-ranking examples of the two or more examples, and at least a portion of an encoder of the neural network is unchanged during the additional training of the neural network; and
translate, based at least in part on the additionally trained neural network, the source text from the first language to the second language.

16. The non-transitory computer-readable medium of claim 15 , wherein the applicability is determined based at least in part on n-grams in the two or more examples of translation that correspond to n-grams in the source text.

17. The non-transitory computer-readable medium of claim 15 , comprising further instructions that, in response to execution by the at least one processor, cause the computing device to at least:
calculate a learning rate based, at least in part, on a measure of the applicability of the one or more of the two or more examples to one or more portions of the source text;
calculate a number of epochs based, at least in part, on the measure of applicability; and perform the additional training based, at least in part, on the learning rate and the number of epochs.

18. The non-transitory computer-readable medium of claim 15 , wherein a portion of the neural network is frozen during the additional training.

19. The non-transitory computer-readable medium of claim 15 , where

Assignees

• Amazon Tech Inc

Inventors

• Mathur Prashant
• Dinu Georgiana
• Currey Anna
• Nowell Eric J
• Upadhyay Aakash
• Yao Haiyu
• Federico Marcello
• Al-Onaizan Yaser
• Pokkunuri Rama Krishna Sandeep
• Wang Jian
• Huang Xianglong

Classifications

• G06F40/58 Primary

  Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation · CPC title

• G06N3/08

  Learning methods · CPC title

• G06F40/45 Primary

  Example-based machine translation; Alignment · CPC title

• G06F40/51

  Translation evaluation · CPC title