Domain adaptation for machine learning models

Patent metadata

Field	Value
Publication number	US-11443193-B2
Application number	US-202016865605-A
Country	US
Kind code	B2
Filing date	May 4, 2020
Priority date	Apr 24, 2020
Publication date	Sep 13, 2022
Grant date	Sep 13, 2022

Abstract

Adapting a machine learning model to process data that differs from training data used to configure the model for a specified objective is described. A domain adaptation system trains the model to process new domain data that differs from a training data domain by using the model to generate a feature representation for the new domain data, which describes different content types included in the new domain data. The domain adaptation system then generates a probability distribution for each discrete region of the new domain data, which describes a likelihood of the region including different content described by the feature representation. The probability distribution is compared to ground truth information for the new domain data to determine a loss function, which is used to refine model parameters. After determining that model outputs achieve a threshold similarity to the ground truth information, the model is output as a domain-agnostic model.

First claim

What is claimed is:

1. In a digital medium environment for adapting a machine learning model to a new domain, a method implemented by at least one computing device, the method comprising:
obtaining a machine learning model configured to receive an input image defined by a first domain and generate an output that classifies the input image by:
extracting local features that identify different regions of the input image using a local feature network;
ascertaining global features that describe objects included in the different regions of the input image by processing the local features using a global feature network; and
assigning a label to each of the different regions of the input image using the local features and the global features;

training the machine learning model to classify an input image defined by a second domain that is different from the first domain by:
causing the local feature network to generate a feature representation that describes local features of the second domain input image;
generating a probability distribution for each of a plurality of discrete regions of the second domain input image by processing the feature representation using a fully convolutional neural network trained upon a segmentation objective;
computing a loss function by comparing the probability distribution to a ground truth classification for the second domain input image; and
refining at least one parameter of the machine learning model using the loss function; and

outputting the machine learning model with its at least one parameter as the trained machine learning model.

2. The method as recited in claim 1 , wherein the first domain comprises images of documents configured in a specified file type, authored in a particular language, and formatted in a certain layout.

3. The method as recited in claim 1 , wherein the local features and features of the second domain input image include vector graphics, text elements, and raster graphics.

4. The method as recited in claim 1 , wherein assigning the label to each of the different regions of the input image comprises generating a local context vector for each of the different regions by processing the local features using a local domain classifier trained to align image features with a local feature alignment objective.

5. The method as recited in claim 1 , wherein the local feature network is configured as a faster region convolutional neural network configured for object detection.

6. The method as recited in claim 1 , wherein assigning the label to each of the different regions of the input image comprises generating a global context vector for each of the different regions by processing the global features using a global domain classifier trained to predict a domain of the input image.

7. The method as recited in claim 1 , further comprising refining at least one parameter of the fully convolutional neural network used to generate the probability distribution for each of the plurality of discrete regions of the second domain input image using the loss function.

8. The method as recited in claim 1 , wherein the second domain input image comprises an image of a document, the method further comprising obtaining the document, extracting metadata from the document, and generating the ground truth classification for the second domain input image using the extracted metadata.

9. The method as recited in claim 1 , wherein the ground truth classification for the second domain input image comprises a labeled version of the second domain input image that includes at least one of:
a vector graphic bounding box identifying a vector graphic in the second domain input data;
a text bounding box identifying text in the second domain input data; or
a raster graphic bounding box identifying a raster graphic in the second domain input data.

10. The method as recited in claim 1 , wherein training the machine learning model to classify the second domain input image further comprises:
receiving a different input image defined by the first domain;
causing the local feature network to generate a feature representation that describes local features of the different first domain input image;
generating a probability distribution for each of a plurality of discrete regions of the different first domain input image by processing the feature representation using the fully convolutional neural network trained upon the segmentation objective; and
updating the loss function by comparing the probability distribution to a ground truth classification for the different first domain input image.

11. In a digital medium environment for adapting a document classification model to a new domain, a method implemented by at least one computing device, the method comprising:
adapting the document classification model to the new domain by:
identifying a data type of training data used to generate the document classification model;
obtaining a new domain document having a data type that is different from the data type of the training data;
causing the document classification model to generate a feature representation describing a plurality of different feature channels that each describe different content included in the new domain document;
generating a probability distribution for each pixel of the new domain document based on the feature representation, the probability distribution describing a likelihood of the pixel depicting each of the plurality of different feature channels;
determining a loss function by comparing the probability distribution to a ground truth for the new domain document; and
refining one or more parameters of the document classification model using the loss function; and

outputting the document classification model with the one or more parameters as a domain-agnostic document classification model.

12. The method as recited in claim 11 , wherein the training data comprises a rasterized image of a document and the document classification model is configured to output at least one bounding box for the rasterized image with a label describing content enclosed by the at least one bounding box.

13. The method as recited in claim 12 , wherein the label describes at least one of paragraph content, list item content, header content, figure content, table content, or background content.

14. The method as recited in claim 11 , wherein the feature channels described by the feature representation include a vector graphic feature channel, a text feature channel, and a raster graphic feature channel.

15. The method as recited in claim 11 , wherein the training data comprises a document authored in English and the new domain document is authored in a language other than English.

16. The method as recited in claim 11 , wherein the ground truth comprises a labeled version of the new domain document that includes at least one of:
a vector graphic bounding box identifying a vector graphic in the new domain document;
a text bounding box identifying text in the new domain document; or
a raster graphic bounding box identifying a raster graphic in the new domain document.

17. The method as recited in claim 11 , further comprising repeating the obtaining, the causing, the generating, the determining, and the refining using at least one additional new domain document until determining that a difference between the probability distribution for the additional new domain document and the ground truth for the additional new domain document satisfies a difference threshold.

18. The method as recited in claim 11 , wherein the docu

Assignees

• Adobe Inc

Inventors

• Li Kai
• Tensmeyer Christopher Alan
• Wigington Curtis Michael
• Zhao Handong
• Barmpalios Nikolaos
• Sun Tong
• Manjunatha Varun
• Morariu Vlad Ion

Classifications

• G06V30/19173

  Classification techniques · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/25

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title

• G06V30/413 Primary

  Classification of content, e.g. text, photographs or tables · CPC title

• G06N3/084 Primary

  Backpropagation, e.g. using gradient descent · CPC title