Domain adaption learning system

Patent metadata

Field	Value
Publication number	US-11620527-B2
Application number	US-201916262878-A
Country	US
Kind code	B2
Filing date	Jan 30, 2019
Priority date	Feb 6, 2018
Publication date	Apr 4, 2023
Grant date	Apr 4, 2023

Abstract

Described is a system for adapting a deep convolutional neural network (CNN). A deep CNN is first trained on an annotated source image domain. The deep CNN is adapted to a new target image domain without requiring new annotations by determining domain agnostic features that map from the annotated source image domain and a target image domain to a joint latent space, and using the domain agnostic features to map the joint latent space to annotations for the target image domain.

First claim

What is claimed is:

1. A system for adapting a deep convolutional neural network (CNN), the system comprising:
one or more processors and a non-transitory computer-readable medium having executable instructions encoded thereon such that when executed, the one or more processors perform operations of:
training a deep CNN on an annotated source image domain;
adapting the deep CNN to a new target image domain without requiring new annotations by determining domain agnostic features that map from the annotated source image domain and a target image domain to a joint latent space and using the domain agnostic features to map the joint latent space to annotations for the target image domain;
wherein the joint latent space is invariant to any structured noise variations between the annotated source image domain and the target image domain; and
wherein decoders add back structured noise variations for reconstructing each image domain from its domain agnostic features in the joint latent space.

2. The system as set forth in claim 1 , wherein the joint latent space is regularized by a plurality of auxiliary networks and loss functions.

3. The system as set forth in claim 1 , where in using the domain agnostic representations to map the joint latent space to annotations for the target image domain, the one or more processors further perform operations of:
using an adversarial setting in which a discriminator tries to classify if a domain agnostic feature in the joint latent space was generated from the annotated source image domain or the target image domain; and
optimizing a cross entropy loss function that is defined as a number of correct classifications of the discriminator.

4. The system as set forth in claim 1 , where in using the domain agnostic representations to map the joint latent space to annotations for the target image domain, the one or more processors further perform operations of:
encoding an image from its actual domain to the joint latent space via an encoder, wherein the actual domain is one of the annotated source image domain and the target image domain;
decoding the image to the other domain via a decoder, wherein the other domain is the other of the annotated source image domain and the target image domain, such that a synthetic image is generated; and
identifying if the synthetic image belongs to the actual domain or the other domain.

5. The system as set forth in claim 4 , wherein the one or more processors further perform operations of:
encoding the synthetic image back to the joint latent space; and
decoding the synthetic image back to its actual domain.

6. The system as set forth in claim 1 , wherein a device is controlled based on the annotations for the target image domain.

7. The system as set forth in claim 6 , wherein the device is a mechanical component of an autonomous vehicle.

8. A computer implemented method for adapting a deep convolutional neural network (CNN), the method comprising an act of:
causing one or more processers to execute instructions encoded on a non-transitory computer-readable medium, such that upon execution, the one or more processors perform operations of:
training a deep CNN on an annotated source image domain;
adapting the deep CNN to a new target image domain without requiring new annotations by determining domain agnostic features that map from the annotated source image domain and a target image domain to a joint latent space and using the domain agnostic features to map the joint latent space to annotations for the target image domain;
wherein the joint latent space is invariant to any structured noise variations between the annotated source image domain and the target image domain; and
wherein decoders add back structured noise variations for reconstructing each image domain from its domain agnostic features in the joint latent space.

9. The method as set forth in claim 8 , wherein the joint latent space is regularized by a plurality of auxiliary networks and loss functions.

10. The method as set forth in claim 8 , where in using the domain agnostic representations to map the joint latent space to annotations for the target image domain, the one or more processors further perform operations of:
using an adversarial setting in which a discriminator tries to classify if a domain agnostic feature in the joint latent space was generated from the annotated source image domain or the target image domain; and
optimizing a cross entropy loss function that is defined as a number of correct classifications of the discriminator.

11. The method as set forth in claim 8 , where in using the domain agnostic representations to map the joint latent space to annotations for the target image domain, the one or more processors further perform operations of:
encoding an image from its actual domain to the joint latent space via an encoder, wherein the actual domain is one of the annotated source image domain and the target image domain;
decoding the image to the other domain via a decoder, wherein the other domain is the other of the annotated source image domain and the target image domain, such that a synthetic image is generated; and
identifying if the synthetic image belongs to the actual domain or the other domain.

12. The method as set forth in claim 11 , wherein the one or more processors further perform operations of:
encoding the synthetic image back to the joint latent space; and
decoding the synthetic image back to its actual domain.

13. A computer program product for adapting a deep convolutional neural network (CNN), the computer program product comprising:
computer-readable instructions stored on a non-transitory computer-readable medium that are executable by a computer having one or more processors for causing the processor to perform operations of:
training a deep CNN on an annotated source image domain;
adapting the deep CNN to a new target image domain without requiring new annotations by determining domain agnostic features that map from the annotated source image domain and a target image domain to a joint latent space and using the domain agnostic features to map the joint latent space to annotations for the target image domain;
wherein the joint latent space is invariant to any structured noise variations between the annotated source image domain and the target image domain; and
wherein decoders add back structured noise variations for reconstructing each image domain from its domain agnostic features in the joint latent space.

14. The computer program product as set forth in claim 13 , wherein the joint latent space is regularized by a plurality of auxiliary networks and loss functions.

15. The computer program product as set forth in claim 13 , where in using the domain agnostic representations to map the joint latent space to annotations for the target image domain, the one or more processors further perform operations of:
using an adversarial setting in which a discriminator tries to classify if a domain agnostic feature in the joint latent space was generated from the annotated source image domain or the target image domain; and
optimizing a cross entropy loss function that is defined as a number of correct classifications of the discriminator.

16. The computer program product as set forth in claim 13 , where in using the domain agnostic representations to map the joint latent space to annotations for the target image domain, the one or more processors further perform operations of:
encoding an image from its actual domain to the joint latent space via an encoder, wherein the actual domain is o

Assignees

• Hrl Lab Llc

Inventors

• Murez Zachary
• Kolouri Soheil
• Kim Kyungnam
• Rostami Mohammad

Classifications

• G06V10/82 Primary

  using neural networks · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0475

  Generative networks · CPC title

• G06N3/0895

  Weakly supervised learning, e.g. semi-supervised or self-supervised learning · CPC title

• G06N3/094

  Adversarial learning · CPC title