Encoding of training data for training of a neural network

Patent metadata

Field	Value
Publication number	US-12374098-B2
Application number	US-202318301543-A
Country	US
Kind code	B2
Filing date	Apr 17, 2023
Priority date	Apr 20, 2022
Publication date	Jul 29, 2025
Grant date	Jul 29, 2025

Abstract

A method for encoding training data for training of a neural network comprises: obtaining training data including multiple datasets, each dataset comprises images annotated with at least one respective object class, forming, each dataset having an individual background class associated with the object class; encoding the images of the datasets to be associated with their respective individual background class; encoding image patches belonging to annotated object classes to be associated with their respective object class; encoding each of the datasets, to include an ignore attribute (“ignore”) to object classes that are annotated only in the other datasets and to background classes formed for the other datasets of the multiple datasets, the ignore attribute indicating that the assigned object class and background classes do not contribute in adapting the neural network in training using the respective dataset; and providing the encoded training data for training of a neural network.

First claim

The invention claimed is:

1. A method for encoding training data for training of a neural network, the method comprising:
obtaining training data including multiple datasets where each dataset comprises images annotated with at least one respective object class and where a number of background classes is less than the number of datasets;
forming, for each of the multiple datasets, an individual background class associated with the at least one respective object class annotated in each of the multiple datasets;
encoding the images of the multiple datasets to be associated with their respective individual background class;
encoding image patches belonging to annotated object classes in the images of the multiple datasets to be associated with their respective object class;
encoding each of the multiple datasets, to include an ignore attribute to object classes that are annotated only in the other datasets of the multiple datasets and to background classes formed for the other datasets of the multiple datasets, the ignore attribute indicating that the assigned object class and background classes do not contribute to adapting the neural network in training using the respective dataset; and
providing the encoded training data for training of a neural network.

2. The method according to claim 1 , wherein a single respective individual background class is formed for each of the multiple datasets.

3. The method according to claim 1 , wherein encoding the images comprises assigning the individual background classes to images patches of the respective ones of the multiple datasets.

4. The method according to claim 1 , further comprising:
forming, for each of the multiple datasets, further object classes that are not annotated in the respective dataset but that are annotated in at least one of the other datasets.

5. The method according to claim 4 , wherein the step of encoding comprises:
providing an ignore attribute to the further object classes.

6. The method according to claim 1 , further comprising:
receiving a further dataset comprising images annotated with at least one additional object class;
forming, for each of the multiple datasets, an additional background class associated with the additional object class annotated in the further dataset;
encoding image patches belonging to annotated object classes in the images of the multiple datasets and the further data set to be associated with their respective object class; and
encoding the further dataset to include an ignore attribute to object classes that are annotated only in the other ones of the multiple datasets and to background classes associated with the other ones of the multiple datasets.

7. The method according to claim 1 , wherein forming the individual background classes is performed so that the number of individual background classes is equal to the number of datasets.

8. The method according to claim 1 , further comprising assigning the ignore attributes to images patches of the datasets including object classes or background classes to be ignored.

9. The method according to claim 1 , wherein the ignore attribute is configured to cause weights of the neural network to not be adjusted for object classes or background classes with the ignore attribute.

10. A method for object detection in an image frame, the method comprising:
training an object detector on the training data encoded with the object classes, background classes, and ignore attributes according to claim 1 as ground truth;
receiving an input image;
detecting an object in the input image using the trained object detector.

11. The method according to claim 10 , wherein the step of training comprises training the object detector on the multiple datasets of the encoded training data simultaneously.

12. A control unit comprising circuitry configured to perform a method for encoding training data for training of a neural network, the method comprising:
obtaining training data including multiple datasets where each dataset comprises images annotated with at least one respective object class and where a number of background classes is less than the number of datasets;
forming, for each of the multiple datasets, an individual background class associated with the at least one respective object class annotated in each of the multiple datasets;
encoding the images of the multiple datasets to be associated with their respective individual background class;
encoding image patches belonging to annotated object classes in the images of the multiple datasets to be associated with their respective object class;
encoding each of the multiple datasets, to include an ignore attribute to object classes that are annotated only in the other datasets of the multiple datasets and to background classes formed for the other datasets of the multiple datasets, the ignore attribute indicating that the assigned object class and background classes do not contribute to adapting the neural network in training using the respective dataset; and
providing the encoded training data for training of a neural network.

13. The control unit according to claim 12 , further comprising:
an image capturing device configured to capture a video stream.

14. The control unit according to claim 13 , wherein the image capturing device is a video camera.

15. A non-transitory computer readable storage medium comprising a computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out a method for encoding training data for training of a neural network, the method comprising:
obtaining training data including multiple datasets where each dataset comprises images annotated with at least one respective object class and where a number of background classes is less than the number of datasets;
forming, for each of the multiple datasets, an individual background class associated with the at least one respective object class annotated in each of the multiple datasets;
encoding the images of the multiple datasets to be associated with their respective individual background class;
encoding image patches belonging to annotated object classes in the images of the multiple datasets to be associated with their respective object class;
encoding each of the multiple datasets, to include an ignore attribute to object classes that are annotated only in the other datasets of the multiple datasets and to background classes formed for the other datasets of the multiple datasets, the ignore attribute indicating that the assigned object class and background classes do not contribute to adapting the neural network in training using the respective dataset; and
providing the encoded training data for training of a neural network.

Assignees

• Axis Ab

Inventors

• Liu Haochen
• Chen Jiandan

Classifications

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06V10/774 Primary

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V10/82 Primary

  using neural networks · CPC title