Hamming distance based robust output encoding for improved generalization

Patent metadata

Field	Value
Publication number	US-11410043-B2
Application number	US-201916413988-A
Country	US
Kind code	B2
Filing date	May 16, 2019
Priority date	May 16, 2019
Publication date	Aug 9, 2022
Grant date	Aug 9, 2022

Abstract

A computer-implemented method generates a hamming code based target label for each class of a dataset in which hamming distance between the target labels in the dataset is maximized and trains a convolutional neural network with the hamming codes based target label to thereby produce a trained AI model. The confusability between classes of the dataset is determined using a confusion matrix. The hamming distances of classes of the dataset that are determined to be more confusable are set to higher values than the hamming distances of classes of the dataset that are determined to be less confusable.

First claim

What is claimed is:

1. A computer-implemented method comprising:
generating a hamming code based target label for each class of a dataset in which hamming distance between the target labels in the dataset is maximized; and
training a convolutional neural network with the modified dataset to thereby produce a trained AI model,
wherein confusability between classes of the dataset is determined using a confusion matrix, and
wherein hamming distances of classes of the dataset that are determined to be more confusable are set to higher values than the hamming distances of classes of the dataset that are determined to be less confusable.

2. The computer-implemented method of claim 1 , wherein the confusability between classes of the dataset is determined by reconstructing data for each class using an autoencoder trained using a first class of the classes and determining a reconstruction error for each class other than the first class.

3. The computer-implemented method of claim 1 , wherein the confusability between classes of the dataset is determined by using a data similarity method to compute a confusion matrix.

4. The computer-implemented method of claim 1 , wherein hamming codes are generated by maximizing the hamming distance between target labels; where the target labels are weighted based on the confusability between classes in the dataset, and wherein there is one hamming code per class in the dataset.

5. The computer-implemented method of claim 1 , wherein training is conducted by backpropagation with binary cross entropy loss.

6. The computer-implemented method of claim 1 , further comprising:
forward propagating using the trained AI model;
binarizing top-k logits of a sigmoid layer to produce a binarized code;
computing Euclidean distance from the binarized code to a set of target hamming codes; and
selecting an output label corresponding to a closest hamming code to thereby produce an inference result.

7. The computer-implemented method of claim 1 , wherein the dataset is a dataset of images.

8. A non-transitory computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by one or more processors to cause the one or more processors to perform a method comprising: generating a hamming code based target label for each class of a dataset in which hamming distance between the target labels in the dataset is maximized; and training a convolutional neural network with the modified dataset to thereby produce a trained AI model, wherein confusability between classes of the dataset is determined using a confusion matrix, and wherein the hamming distances of classes of the dataset that are determined to be more confusable are set to higher values than the hamming distances of classes of the dataset that are determined to be less confusable.

9. The non-transitory computer program product of claim 8 , wherein the confusability between classes of the dataset is determined by reconstructing data for each class using an autoencoder trained using a first class of the classes and determining a reconstruction error for each class other than the first class.

10. The non-transitory computer program product of claim 8 , wherein the confusability between classes of the dataset is determined by using a data similarity method to compute a confusion matrix.

11. The non-transitory computer program product of claim 8 , wherein hamming codes are generated by maximizing the hamming distance between target labels; where the target labels are weighted based on the confusability between classes in the dataset, and wherein there is one hamming code per class in the dataset.

12. The non-transitory computer program product of claim 8 , wherein training is conducted by backpropagation with binary cross entropy loss.

13. The non-transitory computer program product of claim 8 , forward propagating using the trained AI model; binarizing top-k logits of a sigmoid layer to produce a binarized code; computing Euclidean distance from the binarized code to a set of target hamming codes; and selecting an output label corresponding to a closest hamming code to thereby produce an inference result.

14. The non-transitory computer program product of claim 8 , wherein the dataset is a dataset of images.

15. A system including one or more processors configured to implement a method comprising:
generating a hamming code based target label for each class of a dataset in which hamming distance between the target labels in the dataset is maximized; and
training a convolutional neural network with the modified dataset to thereby produce a trained AI model,
wherein confusability between classes of the dataset is determined using a confusion matrix, and
wherein hamming distances of classes of the dataset that are determined to be more confusable are set to higher values than the hamming distances of classes of the dataset that are determined to be less confusable.

16. The system of claim 15 , wherein the confusability between classes of the dataset is determined by reconstructing data for each class using an autoencoder trained using a first class of the classes and determining a reconstruction error for each class other than the first class.

17. The system of claim 15 , wherein the confusability between classes of the dataset is determined by using a data similarity method to compute a confusion matrix.

18. The system of claim 15 , wherein hamming codes are generated by maximizing the hamming distance between target labels; where the target labels are weighted based on the confusability between classes in the dataset, and wherein there is one hamming code per class in the dataset.

19. The system of claim 15 , wherein training is conducted by backpropagation with binary cross entropy loss.

20. The system of claim 15 , wherein the method further comprising:
forward propagating using the trained AI model;
binarizing top-k logits of a sigmoid layer to produce a binarized code;
computing Euclidean distance from the binarized code to a set of target hamming codes; and
selecting an output label corresponding to a closest hamming code to thereby produce an inference result.

Assignees

• IBM

Inventors

• Jaiswal Mayoore Selvarasa
• Cho Minsik
• Kang Bumsoo

Classifications

• G06V10/82

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06N3/048

  Activation functions · CPC title

• G06F18/2413

  based on distances to training or reference patterns · CPC title

• G06N3/045 Primary

  Combinations of networks · CPC title