Domain separation neural networks

Patent metadata

Field	Value
Publication number	US-10970589-B2
Application number	US-201616321189-A
Country	US
Kind code	B2
Filing date	Jul 28, 2016
Priority date	Jul 28, 2016
Publication date	Apr 6, 2021
Grant date	Apr 6, 2021

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing images using an image processing neural network system. One of the system includes a shared encoder neural network implemented by one or more computers, wherein the shared encoder neural network is configured to: receive an input image from a target domain; and process the input image to generate a shared feature representation of features of the input image that are shared between images from the target domain and images from a source domain different from the target domain; and a classifier neural network implemented by the one or more computers, wherein the classifier neural network is configured to: receive the shared feature representation; and process the shared feature representation to generate a network output for the input image that characterizes the input image.

First claim

What is claimed is:

1. A system comprising:
a shared encoder neural network implemented by one or more computers, wherein the shared encoder neural network is configured to:
receive an input image from a target domain; and
process the input image to generate a shared feature representation of features of the input image that are shared between images from the target domain and images from a source domain different from the target domain, wherein the shared encoder neural network has been trained to generate a shared feature representation for the input image from the target domain that, when combined with a private feature representation for the same input image from the target domain generated by a private target encoder neural network that is specific to the target domain, can be used to accurately reconstruct the input image by a shared decoder neural network; and

a classifier neural network implemented by the one or more computers, wherein the classifier neural network is configured to:
receive the shared feature representation; and
process the shared feature representation to generate a network output for the input image that characterizes the input image.

2. The system of claim 1 , wherein images from the target domain have different low level image statistics than images from images from the source domain.

3. The system of claim 1 , wherein the network output is an object classification output.

4. The system of claim 1 , wherein the network output is a pose estimation output.

5. The system of claim 1 , wherein classifier neural network has been trained on labeled images from the source domain.

6. The system of claim 1 , wherein the shared encoder neural network has been trained to generate shared feature representations for input images from the target domain that are similar to shared feature representations for input images from the source domain.

7. The system of claim 6 , wherein the shared encoder neural network has been trained to generate shared feature representations for input images from the target domain that are different from private feature representations for the same input images from the target domain generated by the private target encoder neural network that is specific to the target domain.

8. The system of claim 7 , wherein the shared encoder neural network has been trained to generate shared feature representations for input images from the source domain that are different from private feature representations for the same input images from the source domain generated by a private source encoder neural network that is specific to the source domain.

9. The system of claim 7 , wherein the shared encoder neural network has been trained to generate a shared feature representation for an input image from the source domain that, when combined with a private feature representation for the same input image generated by the private source encoder neural network, can be used to accurately reconstruct the input image by the shared decoder neural network.

10. The system of claim 9 , wherein the shared encoder neural network, the private source encoder neural network, and the private target encoder neural network are convolutional neural networks with a same architecture but different parameter values.

11. The system of claim 1 , wherein the classifier neural network is a fully-connected neural network.

12. A non-transitory computer-readable storage medium encoded with instructions that, when executed by one or more computers, cause the one or more computers to implement:
a shared encoder neural network, wherein the shared encoder neural network is configured to:
receive an input image from a target domain; and
process the input image to generate a shared feature representation of features of the input image that are shared between images from the target domain and images from a source domain different from the target domain, wherein the shared encoder neural network has been trained to generate a shared feature representation for the input image from the target domain that, when combined with a private feature representation for the same input image from the target domain generated by a private target encoder neural network that is specific to the target domain, can be used to accurately reconstruct the input image by a shared decoder neural network; and

a classifier neural network, wherein the classifier neural network is configured to:
receive the shared feature representation; and
process the shared feature representation to generate a network output for the input image that characterizes the input image.

13. The non-transitory computer-readable storage medium of claim 12 , wherein images from the target domain have different low level image statistics than images from images from the source domain.

14. The non-transitory computer-readable storage medium of claim 12 , wherein the network output is an object classification output or a pose estimation output.

15. A method performed by one or more computers, the method comprising:
receiving an input image from a target domain;
processing the input image from the target domain using a shared encoder neural network, wherein the shared encoder neural network is configured to:
receive the input image from the target domain; and
process the input image to generate a shared feature representation of features of the input image that are shared between images from the target domain and images from a source domain different from the target domain, wherein the shared encoder neural network has been trained to generate a shared feature representation for the input image from the target domain that, when combined with a private feature representation for the same input image from the target domain generated by a private target encoder neural network that is specific to the target domain, can be used to accurately reconstruct the input image by a shared decoder neural network; and

processing the input image from the target domain using a classifier neural network, wherein the classifier neural network is configured to:
receive the shared feature representation; and
process the shared feature representation to generate a network output for the input image that characterizes the input image.

16. The method of claim 15 , wherein images from the target domain have different low level image statistics than images from images from the source domain.

17. The method of claim 15 , wherein the network output is an object classification output.

18. The method of claim 15 , wherein the network output is a pose estimation output.

19. The method of claim 15 , wherein classifier neural network has been trained on labeled images from the source domain.

20. The method of claim 15 , wherein the shared encoder neural network has been trained to generate shared feature representations for input images from the target domain that are similar to shared feature representations for input images from the source domain.

21. The method of claim 20 , wherein the shared encoder neural network has been trained to generate shared feature representations for input images from the target domain that are different from private feature representations for the same input images from the target domain generated by the private target encoder neural network that is specific to the target domain.

22. The method of claim 21 , wherein the shared encoder neural network has been trained to generate shared feature representations for input images fr

Assignees

• Google Llc

Inventors

• Bousmalis Konstantinos
• Silberman Nathan
• Krishnan Dilip
• Trigeorgis George
• Erhan Dumitru

Classifications

• G06N3/084 Primary

  Backpropagation, e.g. using gradient descent · CPC title

• G06V10/72

  Data preparation, e.g. statistical preprocessing of image or video features · CPC title

• G06V10/7715

  Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods · CPC title

• G06F18/241

  relating to the classification model, e.g. parametric or non-parametric approaches · CPC title

• G06N3/045

  Combinations of networks · CPC title