AI image recognition training tool sets

Patent metadata

Field	Value
Publication number	US-11263482-B2
Application number	US-201916537363-A
Country	US
Kind code	B2
Filing date	Aug 9, 2019
Priority date	Aug 9, 2019
Publication date	Mar 1, 2022
Grant date	Mar 1, 2022

Abstract

Systems and methods to label images for inclusion into a machine learning image recognition training data set. A coded labeling definition is defined. A set of digital images is collected that includes images of different types of piece of equipment. A digital image is presented that includes an image of a particular piece of equipment. A received numeric code corresponding to the particular piece of equipment within the present image is received. The received numeric code is associated with the present image and the present image is stored in association with the received numeric code into a machine learning image recognition data set. A machine learning image recognition is trained based on the machine learning image recognition data set to automatically associate unlabeled images with the respective numeric code that corresponds to the particular piece of equipment in the unlabeled image.

First claim

What is claimed is:

1. A method for labeling images for inclusion into a machine learning image recognition training data set, the method comprising:
defining a coded labeling definition for labeling images capturing views of equipment that are within a defined number of types of pieces of equipment, the coded labeling definition comprising a respective numeric code corresponding to each respective type of each piece of equipment within the defined number of types of pieces of equipment,
wherein each respective numeric code comprises a respective first subcode and a respective second subcode, and
wherein each respective numeric code comprises a separator character separating the respective first subcode and the respective second subcode;

collecting a set of digital images comprising a plurality of images of different types of piece of equipment within the defined number of types of pieces of equipment;
presenting, on a user interface device;
a present image from within the set of digital images, the present image comprising an image of a particular piece of equipment wherein the particular piece of equipment corresponds to a particular type of a piece of equipment within the defined number of types of pieces of equipment; and
an input interface comprising a first input field for receiving a first subcode input and a second input field for receiving a second subcode input, and further presents the separator character between the first input field and the second input field;

receiving user input data via the user interface device, where the user input data is a specification by a user of a received numeric code within the coded labeling definition that corresponds to the particular type of the particular piece of equipment within the present image,
wherein the received numeric code comprises the first subcode input corresponding to the respective first subcode and the second subcode input corresponding to the respective second subcode;

associating, based on presenting the image and receiving the received numeric code, the received numeric code with the present image;
storing the present image in association with the received numeric code into a machine learning image recognition data set; and
training a machine learning image recognition process by processing, with a computer processor, the machine learning image recognition data set, wherein training the machine learning image recognition process comprises training to automatically associate unlabeled images comprising an image the particular type of the piece of equipment with the respective numeric code within the coded labeling definition that corresponds to the particular type of the piece of equipment.

2. The method of claim 1 , further comprising:
providing, to a user of the user interface device, a reference catalog, the reference catalog comprising, for each respective type of each piece of equipment within the defined number of types of pieces of equipment, a respective presentation that comprises:
at least one reference image of a respective type of a piece of equipment presented in the respective presentation; and
a respective numeric code within the coded labeling definition that corresponds to the particular type of the piece of equipment presented in the respective presentation.

3. The method of claim 1 , wherein the respective first subcode indicating a category of types of pieces of equipment, and the respective second subcode indicating a subcategory of types of pieces of equipment in the respective first subcode.

4. The method of claim 3 , wherein each respective first subcode comprises a first number of digits, and wherein each respective second subcode comprises a second number of digits.

5. The method of claim 4 , wherein each respective first subcode contains either two or three digits, and
wherein each respective second subcode contains either two or three digits.

6. A system for labeling images for inclusion into a machine learning image recognition training data set, the system comprising:
at least one processor;
a memory communicatively coupled to the processor;
a user interface;
the at least one processor, when operating, being configured to:
maintain a definition of a coded labeling definition for labeling images capturing views of equipment that are within a defined number of types of pieces of equipment, the coded labeling definition comprising a respective numeric code corresponding to each respective type of each piece of equipment within the defined number of types of pieces of equipment,
wherein each respective numeric code comprises a respective first subcode and a respective second subcode, and
wherein each respective numeric code comprises a separator character separating the respective first subcode and the respective second subcode;

store a set of digital images comprising a plurality of images of different types of piece of equipment within the defined number of types of pieces of equipment;
present, on the user interfaced;
a present image from within the set of digital images, the present image comprising an image of a particular piece of equipment wherein the particular piece of equipment corresponds to a particular type of a piece of equipment within the defined number of types of pieces of equipment; and
an input interface comprising a first input field for receiving a first subcode input and a second input field for receiving a second subcode input, and further presents the separator character between the first input field and the second input field;

receive user input data via the user interface, where the user input data is a specification by a user of a received numeric code within the coded labeling definition that corresponds to the particular type of the particular piece of equipment within the present image;
wherein the received numeric code comprises the first subcode input corresponding to the respective first subcode and the second subcode input corresponding to the respective second subcode;

associate, based on presentation of the image and receipt of the received numeric code, the received numeric code with the present image;
store the present image in association with the received numeric code into a machine learning image recognition data set; and
train a machine learning image recognition process by processing the machine learning image recognition data set, wherein training of the machine learning image recognition process comprises training to automatically associate unlabeled images comprising an image the particular type of the piece of equipment with the respective numeric code within the coded labeling definition that corresponds to the particular type of the piece of equipment.

7. The system of claim 6 , wherein the at least one processor is further configured to:
provide, to the user interface, a reference catalog, the reference catalog comprising, for each respective type of each piece of equipment within the defined number of types of pieces of equipment, a respective presentation that comprises:
at least one reference image of a respective type of a piece of equipment presented in the respective presentation; and
a respective numeric code within the coded labeling definition that corresponds to the particular type of the piece of equipment presented in the respective presentation.

8. The system of claim 6 , wherein the respective first subcode indicating a category of types of pieces of equipment, and the respective second subcode indicating a subcategory of types of pieces of equipment in the respective first subcode.

9. The system of claim 8 , wherein each respective first subcode comprises a first number of digits, and wherein each respective second subcode comprises

Assignees

• Florida Power & Light Co

Inventors

• Schwartz Eric D
• Du Yinuo
• Epel Aaron J
• Barbosa Daniel J
• Deangelis Melanie A
• Bracci Griffin M

Classifications

• G06N3/105 Primary

  Shells for specifying net layout · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06F18/40

  Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor · CPC title