Network reparameterization for new class categorization

Patent metadata

Field	Value
Publication number	US-11087184-B2
Application number	US-201916580199-A
Country	US
Kind code	B2
Filing date	Sep 24, 2019
Priority date	Sep 25, 2018
Publication date	Aug 10, 2021
Grant date	Aug 10, 2021

Abstract

A computer-implemented method and system are provided for training a model for New Class Categorization (NCC) of a test image. The method includes decoupling, by a hardware processor, a feature extraction part from a classifier part of a deep classification model by reparametrizing learnable weight variables of the classifier part as a combination of learnable variables of the feature extraction part and of a classification weight generator of the classifier part. The method further includes training, by the hardware processor, the deep classification model to obtain a trained deep classification model by (i) learning the feature extraction part as a multiclass classification task, and (ii) episodically training the classifier part by learning a classification weight generator which outputs classification weights given a training image.

First claim

What is claimed is:

1. A computer-implemented method for training a model for New Class Categorization (NCC) of a test image, comprising:
decoupling, by a hardware processor, a feature extraction part from a classifier part of a deep classification model by reparametrizing learnable weight variables of the classifier part as a combination of learnable variables of the feature extraction part and of a classification weight generator of the classifier part; and
training, by the hardware processor, the deep classification model to obtain a trained deep classification model by (i) learning the feature extraction part as a multiclass classification task, and (ii) episodically training the classifier part by learning a classification weight generator which outputs classification weights given a training image.

2. The computer-implemented method of claim 1 , wherein the model for the NCC of the test image is trained given limited exemplar class information below a threshold amount.

3. The computer-implemented method of claim 2 , wherein the limited exemplar class information comprises a number of semantic attributes below the threshold amount.

4. The computer-implemented method of claim 2 , wherein the limited exemplar class information comprises a number of labeled examples below the threshold amount.

5. The computer-implemented method of claim 1 , further comprising:
performing feature extraction to obtain one or more discriminative feature representations for the test image; and
classifying the test image as a new class relative to a set of known classes used to train the classification model based on a classification weight determined for the test image.

6. The computer-implemented method of claim 1 , wherein said learning step comprises training the multiclass classification task to distinguish between all classes within a training data set.

7. The computer-implemented method of claim 1 , wherein said training step comprising training the classification weight generator using a cosine similarity based softmax function.

8. The computer-implemented method of claim 1 , wherein in said training step, the learning and episodically training steps are performed independently.

9. The computer-implemented method of claim 1 , wherein the method is performed relative to a training set of images and a training set of exemplar class information corresponding to the training set of images.

10. The computer-implemented method of claim 1 , wherein the classifier part guides a learning of the classification weight generator using a softmax function.

11. A computer program product for training a model for New Class Categorization (NCC) of a test image, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method comprising:
decoupling, by a hardware processor of the computer, a feature extraction part from a classifier part of a deep classification model by reparametrizing learnable variables of the classifier part as a combination of learnable variables of the feature extraction part and of a classification weight generator of the classifier part; and
training, by the hardware processor, the deep classification model to obtain a trained deep classification model by (i) learning the feature extraction part as a multiclass classification task, and (ii) episodically training the classifier part by learning a classification weight generator which outputs classification weights given a training image and the limited exemplar class information below the threshold amount corresponding to the training image.

12. The computer program product of claim 11 , wherein the model for the NCC of the test image is trained given limited exemplar class information below a threshold amount.

13. The computer program product of claim 12 , wherein the limited exemplar class information comprises a number of semantic attributes below the threshold amount.

14. The computer program product of claim 12 , wherein the limited exemplar class information comprises a number of labeled examples with a total number below the threshold amount.

15. The computer program product of claim 11 , wherein the method further comprises:
performing feature extraction to obtain one or more discriminative feature representations for the test image; and
classifying the test image as a new class relative to a set of known classes used to train the classification model based on a classification weight determined for the test image from at least the one or more discriminative feature representations.

16. The computer program product of claim 11 , wherein said training step comprising training the classification weight generator using a cosine similarity based softmax function.

17. The computer program product of claim 11 , wherein in said training step, the learning and episodically training steps are performed independently.

18. The computer program product of claim 11 , wherein the method is performed relative to a training set of images and a training set of exemplar class information corresponding to the training set of images.

19. The computer program product of claim 11 , wherein the classifier part guides a learning of the classification weight generator using a softmax function.

20. A computer processing system for training a model for New Class Categorization (NCC) of a test image, comprising:
a memory device including program code stored thereon;
a hardware processor, operatively coupled to the memory device, and configured to run the program code stored on the memory device to
decouple a feature extraction part from a classifier part of a deep classification model by reparametrizing learnable variables of the classifier part as a combination of learnable variables of the feature extraction part and of a classification weight generator of the classifier part; and
train the deep classification model to obtain a trained deep classification model by (i) learning the feature extraction part as a multiclass classification task, and (ii) episodically training the classifier part by learning a classification weight generator which outputs classification weights given a training image.

Assignees

• Nec Lab America Inc
• Nec Corp

Inventors

• Min Renqiang
• Li Kai
• Bai Bing
• Graf Hans Peter

Classifications

• G06V10/454 Primary

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/2431

  Multiple classes · CPC title

• G06F18/2148

  characterised by the process organisation or structure, e.g. boosting cascade · CPC title