System and method for classification of ambiguous objects

Patent metadata

Field	Value
Publication number	US-11748669-B2
Application number	US-202218074187-A
Country	US
Kind code	B2
Filing date	Dec 2, 2022
Priority date	Mar 27, 2020
Publication date	Sep 5, 2023
Grant date	Sep 5, 2023

Abstract

The method for classifying ambiguous objects, including: determining initial labels for an image set; determining N training sets from the initially-labelled image set; training M annotation models using the N training sets; determining secondary labels for each image of the image set using the M trained annotation models; and determining final labels for the image set based on the secondary labels. The method can optionally include training a runtime model using images from the image set labeled with the final labels; and optionally using the runtime model.

First claim

We claim:

1. A method for ambiguous object classification, comprising:
receiving an image set, wherein each image of the image set is labeled with a noisy label of an ambiguous object class;
partitioning the image set into N training sets;
training annotation models using the N training sets;
generating a set of secondary labels for each image of the image set using the trained annotation models; and
determining a final label for images of the image set based on the respective set of secondary labels.

2. The method of claim 1 , wherein the N training sets are split into K orders, wherein each order comprises all images of the image set.

3. The method of claim 2 , wherein training sets belonging to the same order are non-overlapping.

4. The method of claim 2 , wherein training sets belonging to the same order are the same size.

5. The method of claim 1 , further comprising training a runtime model using the image set associated with the final labels.

6. The method of claim 5 , further comprising:
selecting an operation program based on a determined classification from the runtime model; and
operating an appliance according to the operation program.

7. The method of claim 1 , wherein the image set is received from a set of appliances.

8. The method of claim 1 , wherein the image set comprises images that depict a view from above a scene.

9. The method of claim 1 , wherein the ambiguous object class comprises a food type.

10. The method of claim 1 , wherein each of the annotation models is trained using a different training set of the N training sets.

11. The method of claim 1 , wherein the set of secondary labels for a given image is generated using the trained annotation models that were not trained using the image.

12. The method of claim 1 , wherein when more than a threshold number of secondary labels disagree for a given image, determining the final label for the image comprises facilitating reannotation of the image.

13. The method of claim 1 , further comprising removing an image from the image set when more than a threshold number of secondary labels for the image disagree.

14. The method of claim 1 , wherein the final label for a given image is determined based on a majority vote between the secondary labels within the set of secondary labels for the image.

15. A non-transitory computer-readable storage medium storing instructions that, when executed by a processing system, cause the processing system to perform a method comprising:
receiving an image set, wherein each image of the image set is labeled with a noisy label of an ambiguous object class;
partitioning the image set into N training sets;
training annotation models using the N training sets;
generating a set of secondary labels for each image of the image set using the trained annotation models;
determining a final label for each image based on the respective set of secondary labels; and
training a runtime model using the image set and the final labels.

16. The non-transitory computer-readable storage medium of claim 15 , wherein the method further comprises:
receiving an inference image from an appliance;
selecting an operation program based on a determined classification for the inference image from the runtime model; and
operating the appliance according to the operation program.

17. The non-transitory computer-readable storage medium of claim 15 , wherein the N training sets are split into K orders, wherein each order comprises all images of the image set.

18. The non-transitory computer-readable storage medium of claim 15 , wherein the final label for a given image is determined based on agreement between a threshold number of secondary labels for the image.

Assignees

• June Life Inc

Inventors

• Bhogal Nikhil
• Kumar Nishit
• Paruchuri Jithendra

Classifications

• G06N3/091

  Active learning · CPC title

• G06N3/0895

  Weakly supervised learning, e.g. semi-supervised or self-supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N20/20 Primary

  Ensemble learning · CPC title