Estimation of parameters for machine translation without in-domain parallel data

Patent metadata

Field	Value
Publication number	US-9652453-B2
Application number	US-201414252032-A
Country	US
Kind code	B2
Filing date	Apr 14, 2014
Priority date	Apr 14, 2014
Publication date	May 16, 2017
Grant date	May 16, 2017

Abstract

A system and method for estimating parameters for features of a translation scoring function for scoring candidate translations in a target domain are provided. Given a source language corpus for a target domain, a similarity measure is computed between the source corpus and a target domain multi-model, which may be a phrase table derived from phrase tables of comparative domains, weighted as a function of similarity with the source corpus. The parameters of the log-linear function for these comparative domains are known. A mapping function is learned between similarity measure and parameters of the scoring function for the comparative domains. Given the mapping function and the target corpus similarity measure, the parameters of the translation scoring function for the target domain are estimated. For parameters where a mapping function with a threshold correlation is not found, another method for obtaining the target domain parameter can be used.

First claim

What is claimed is:

1. A method for estimating parameters for features of a translation scoring function and for scoring candidate translations in a target domain comprising:
receiving a monolingual source corpus for a target domain and deriving n-gram counts from the monolingual source corpus or receiving n-gram counts derived only from the monolingual source corpus, the monolingual source corpus comprising sentences in a source language;
generating a multi-model for the target domain based on a phrase table for each of a set of comparative domains and a measure of similarity between the n-gram counts derived only from the source corpus for the target domain and the phrase tables for the comparative domains, each of the phrase tables storing a value for each of a set of features for each of a set of biphrases, the generated target domain multi-model being a weighted combination of two or more of the phrase tables for the comparative domains;
for the target domain, computing a measure of similarity between the monolingual source corpus and the target domain multi-model;
for each of a plurality of the comparative domains, computing a measure of similarity between a source corpus for the comparative domain and a respective comparative domain multi-model that is derived from phrase tables for others of the set of the comparative domains, each of the plurality of comparative domains being associated with parameters for at least some of the features of the translation scoring function;
estimating the parameters of the translation scoring function for the target domain based on the computed measure of similarity between the source corpus and the target domain multi-model, the computed measures of similarity for the comparative domains, and the parameters for the scoring function for the comparative domains; and
with a statistical machine translation component, scoring a translation with the translation scoring function,
wherein the generating of the target domain multi-model, computing the measure of similarity between the source corpus and the target domain multi-model, computing the measure of similarity between a source corpus for the comparative domains and the respective comparative domain multi-models, and the estimating the parameters for the translation scoring function are performed with a computer processor.

2. The method of claim 1 , wherein the estimating of the parameters comprises:
learning a function which maps values of at least one parameter of the translation scoring function to the computed measures of similarity for the comparative domains; and
where the learned function indicates a correlation between the at least one parameter and the computed measures of similarity, estimating the at least one parameter for the target domain based on the learned function.

3. The method of claim 2 , where the learned function is a linear regression function.

4. The method of claim 2 , wherein when a predefined correlation is not found, estimating the at least one parameter for the translation scoring function based on the corresponding at least one parameter of one of the comparative domains that has a computed similarity with the respective comparative domain multi-model which is closest to the computed similarity with the target domain multi-model.

5. The method of claim 1 , wherein each similarity measure is computed as a function of counts of n-grams of each of a plurality of sizes in the source corpus of the respective domain that are present in the phrase table or multi-model with which the similarity is being computed.

6. The method of claim 5 , wherein each similarity measure may be computed as a function of

(

∏

n
=
p

n
=
q

⁢

⁢

match
⁡

(

n
|
pt

,
s

)

total
⁡

(

n
|
s

)

)

1
r

,

where match(n|pt,s) is the count of n-grams of order n in the monolingual source corpus s that exist in a source side of a respective phrase table pt,
total(n|s) is the number of n-grams of order n in the source corpus, and
p is a first value of n, q is a second value of n higher than p, and r is the total number of values of n used in the computation.

7. The method of claim 1 , wherein the generating of the multi-model for the target domain comprises combining the phrase tables for the comparative domains in a weighted combination in which each of the comparative domain phrase tables is weighted as a function of the measure of similarity between the source corpus for the target domain and the comparative domain phrase table.

8. The method of claim 1 , wherein the multi-model for a first of the comparative domains is generated by combining the phrase tables for others of the comparative domains in a weighted combination in which each of the other comparative domain phrase tables is weighted as a function of the measure of similarity between the source corpus for the first comparative domain and the other comparative domain phrase table.

9. The method of claim 1 , wherein the method is performed without access to a parallel corpus in the target domain.

10. The method of claim 1 , wherein the set of comparative domains comprises at least three comparative domains.

11. The method of claim 1 , wherein the translation scoring function is a log-linear scoring function.

12. The method of claim 1 , wherein the features of the translation scoring function include features selected from the group consisting of lexical features, phrasal features, reordering features, and language model features.

13. The method of claim 12 , wherein the features of the translation scoring function include lexical features, phrasal features, reordering features, and at least one language model feature.

14. The method of claim 1 , wherein each of the comparative domain phrase tables includes biphrase features for each of a set of biphrases, each biphrase including a source phrase and a corresponding target phrase, the biphrase features having been derived from a parallel corpus of source and target text strings.

15. A computer program product

Assignees

• Xerox Corp

Inventors

• Mathur Prashant
• Venkatapathy Sriram
• Cancedda Nicola

Classifications

• G06F40/44 Primary

  Statistical methods, e.g. probability models · CPC title

• G06F16/3344 Primary

  using natural language analysis · CPC title

• G06F17/2818 Primary

  Physics · mapped topic

• G06F17/30684

  Physics · mapped topic