Systems and methods for part identification and assessment using multiple images

Patent metadata

Field	Value
Publication number	US-11604940-B2
Application number	US-202017123051-A
Country	US
Kind code	B2
Filing date	Dec 15, 2020
Priority date	Dec 15, 2020
Publication date	Mar 14, 2023
Grant date	Mar 14, 2023

Abstract

A method for object identification using multiple images. The method includes training an object identification model. Training the model includes collecting a training images for each of a plurality of objects, labeling each of the plurality of training images with a corresponding one of a plurality of object identifiers, and training a neural network with the plurality of labeled training images. At least two target images of a target object are receive and fed into the trained object identification model. The method further includes receiving, from the trained object identification model, for each of the at least two target images, an object identifier corresponding to the target object and a probability that the object identifier corresponds to the target object. A similarity value between the at least two target images is computed and the probabilities for the at least two target images are combined in proportion to the similarity value.

First claim

What is claimed is:

1. A method for object identification using multiple images, comprising: training an object identification model, including: collecting a plurality of training images for each of a plurality of objects; labeling each of the plurality of training images with a corresponding one of a plurality of object identifiers; and training a neural network with the plurality of labeled training images; receiving at least two target images of a target object; feeding each of the at least two target images into the trained object identification model; receiving, from the trained object identification model, for each of the at least two target images, a target object identifier corresponding to the target object and a probability that the target object identifier corresponds to the target object; computing a similarity value between the at least two target images; combining the probabilities for the at least two target images in proportion to the similarity value; and identifying the target object as the target object identifier based on the combined probabilities.

2. The method of claim 1 , wherein collecting the plurality of training images comprises receiving a plurality of photographs for each of the plurality of objects.

3. The method of claim 1 , wherein collecting the plurality of training images comprises rendering a plurality of training images for each of the plurality of objects.

4. The method of claim 3 , wherein rendering the plurality of training images comprises rendering images for each of the plurality of objects as viewed from different angles.

5. The method of claim 1 , further comprising displaying information, including an image of a part corresponding to the target object identifier located on an associated machine.

6. The method of claim 5 , further comprising displaying a set of suitable substitute parts for the part corresponding to the target object identifier.

7. An object identification system, comprising: one or more processors; and one or more memory devices having stored thereon instructions that when executed by the one or more processors cause the one or more processors to: train an object identification model, including: collecting a plurality of training images for each of a plurality of objects; labeling each of the plurality of training images with a corresponding one of a plurality of object identifiers; and training a neural network with the plurality of labeled training images; receive at least two target images of a target object; feed each of the at least two target images into the trained object identification model; receive, from the trained object identification model, for each of the at least two target images, a probability that the target object corresponds to each of the plurality of object identifiers; compute a similarity value between the at least two target images; combine, for each of the plurality of object identifiers, the corresponding probabilities for the at least two target images in proportion to the similarity value; and identify the target object as the object identifier having a greatest combined probability.

8. The system of claim 7 , wherein collecting the plurality of training images comprises receiving a plurality of photographs for each of the plurality of objects.

9. The system of claim 7 , wherein collecting the plurality of training images comprises rendering a plurality of training images for each of the plurality of objects.

10. The system of claim 9 , wherein rendering the plurality of training images comprises rendering images for each of the plurality of objects as viewed from different angles.

11. The system of claim 7 , further comprising normalizing the combined probabilities for each object identifier.

12. The system of claim 7 , wherein the target object is a part associated with a machine and further comprising:
receiving machine information identifying the machine; and
based on the machine information, removing selected object identifiers from the plurality of object identifiers.

13. The system of claim 7 , further comprising displaying information, including an image of a part corresponding to the object identifier having the greatest combined probability located on an associated machine.

14. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising: training an object identification model, including: collecting a plurality of training images for each of a plurality of objects; labeling each of the plurality of training images with a corresponding one of a plurality of object identifiers; and training a neural network with the plurality of labeled training images; receiving at least two target images of a target object; feeding each of the at least two target images into the trained object identification model; receiving, from the trained object identification model, for each of the at least two target images, a target object identifier corresponding to the target object and a probability that the target object identifier corresponds to the target object; computing a similarity value between the at least two target images; combining the probabilities for the at least two target images in proportion to the similarity value; and identifying the target object as the target object identifier based on the combined probabilities.

15. The one or more non-transitory computer-readable media of claim 14 , wherein collecting the plurality of training images comprises receiving a plurality of photographs for each of the plurality of objects.

16. The one or more non-transitory computer-readable media of claim 14 , wherein collecting the plurality of training images comprises rendering a plurality of training images for each of the plurality of objects.

17. The one or more non-transitory computer-readable media of claim 16 , wherein rendering the plurality of training images comprises rendering images for each of the plurality of objects as viewed from different angles.

18. The one or more non-transitory computer-readable media of claim 14 , further comprising displaying information, including an image of a part corresponding to the target object identifier located on an associated machine.

19. The one or more non-transitory computer-readable media of claim 18 , further comprising displaying a set of suitable substitute parts for the part corresponding to the target object identifier.

20. The one or more non-transitory computer-readable media of claim 18 , further comprising displaying ordering information for the part corresponding to the target object identifier.

Assignees

• Caterpillar Inc

Inventors

• Reaume Daniel Joseph
• Radakovic Daniela
• Organ Daniel Jude

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06V20/60

  Type of objects · CPC title

• G06F18/2415

  based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate · CPC title

• G06F18/214 Primary

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title