Systems and methods for image classification

Patent metadata

Field	Value
Publication number	US-11798270-B2
Application number	US-202117216746-A
Country	US
Kind code	B2
Filing date	Mar 30, 2021
Priority date	Apr 27, 2020
Publication date	Oct 24, 2023
Grant date	Oct 24, 2023

Abstract

In one aspect, a method for inspecting features of an image using an image inspection controller that includes a processor communicatively coupled to a memory is described. The method includes receiving, at the processor, an input image, performing, on the input image, one of a semantic segmentation process and an object classification process to generate an output image, and prompting a user to select between approving the displayed output image, and at least one of i) performing an additional semantic segmentation process on the displayed output image, and ii) performing an additional object classification process on the displayed output image.

First claim

What is claimed is:

1. A method for inspecting features of an image using an image inspection controller that includes a processor communicatively coupled to a memory, said method comprising:
receiving, at the processor, an input image;
performing, on the input image, one of a semantic segmentation process and an object classification process to generate an output image;
wherein the semantic segmentation process comprises:
annotating, by the processor, at least one segment of the input image to produce a semantic segmentation annotated image;
generating, by the processor, a semantic segmentation model based on a semantic segmentation training vector derived from the semantic segmentation annotated image;
applying, by the processor, the semantic segmentation model to each pixel of the input image to generate the output image; and
displaying the output image; and

wherein the object classification process comprises:
annotating, by the processor, at least one object in an object mask to produce an object classification annotated image;
generating, by the processor, an object classification model based on an object classification training vector derived from the object classification annotated image;
applying, by the processor, the object classification model to the object mask to generate the output image, and
displaying the output image; and

prompting a user to select between approving and rejecting the displayed output image, and, in response to the user rejecting the displayed output image, performing at least one of i) an additional semantic segmentation process on the displayed output image, and ii) an additional object classification process on the displayed output image.

2. The method in accordance with claim 1 , further comprising receiving user input indicating a selection of performing the additional semantic segmentation process on the displayed output image.

3. The method in accordance with claim 2 , further comprising performing the additional semantic segmentation process on the displayed output image, the additional semantic segmentation process comprising generating and displaying an updated output image.

4. The method in accordance with claim 3 , further comprising prompting the user to select between approving the displayed updated output image, and at least one of i) performing a further semantic segmentation process on the displayed updated output image, and ii) performing a further object classification process on the displayed updated output image.

5. The method in accordance with claim 1 , further comprising receiving user input indicating a selection of performing the additional object classification process on the displayed output image.

6. The method in accordance with claim 5 , further comprising performing the additional object classification process on the displayed output image, the additional object classification process comprising generating and displaying an updated output image.

7. The method in accordance with claim 6 , further comprising prompting the user to select between approving the displayed updated output image, and at least one of i) performing a further semantic segmentation process on the displayed updated output image, and ii) performing a further object classification process on the displayed updated output image.

8. The method in accordance with claim 1 , wherein annotating at least one segment of the input image or annotating at least one object in the object mask comprises annotating the input image or the object mask, based on user input, on a graphical user interface displayed on a display device communicatively coupled to the processor.

9. An image inspection computing device comprising:
a memory device; and
at least one processor communicatively coupled to said memory device, wherein said at least one processor is configured to:
receive an input image;
perform, on the input image, one of a semantic segmentation process and an object classification process to generate an output image;
wherein the semantic segmentation process comprises:
annotating, by the processor, at least one segment of the input image to produce a semantic segmentation annotated image;
generating, by the processor, a semantic segmentation model based on a semantic segmentation training vector derived from the semantic segmentation annotated image;
applying, by the processor, the semantic segmentation model to each pixel of the input image to generate the output image; and
displaying the output image; and
wherein the object classification process comprises:
annotating, by the processor, at least one object in an object mask to produce an object classification annotated image;
generating, by the processor, an object classification model based on an object classification training vector derived from the object classification annotated image;
applying, by the processor, the object classification model to the object mask to generate the output image; and
displaying the output image; and
prompt a user to select between approving and rejecting the displayed output image, and, in response to the user rejecting the displayed output image, performing at least one of i) an additional semantic segmentation process on the displayed output image, and ii) an additional object classification process on the displayed output image.

10. The image inspection computing device in accordance with claim 9 , wherein said processor is further configured to receive user input indicating a selection of performing the additional semantic segmentation process on the displayed output image.

11. The image inspection computing device in accordance with claim 10 , wherein said processor is further configured to perform the additional semantic segmentation process on the displayed output image, the additional semantic segmentation process including generating and displaying an updated output image.

12. The image inspection computing device in accordance with claim 11 , wherein said processor is further configured to prompt the user to select between approving the displayed updated output image, and at least one of i) performing a further semantic segmentation process on the displayed updated output image, and ii) performing a further object classification process on the displayed updated output image.

13. The image inspection computing device in accordance with claim 9 , wherein said processor is further configured to receive user input indicating a selection of performing the additional object classification process on the displayed output image.

14. The image inspection computing device in accordance with claim 13 , wherein said processor is further configured to perform the additional object classification process on the displayed output image, the additional object classification process including generating and displaying an updated revised output image.

15. The image inspection computing device in accordance with claim 14 , wherein said processor is further configured to prompt the user to select between approving the displayed updated output image, and at least one of i) performing a further semantic segmentation process on the displayed updated output image, and ii) performing a further object classification process on the displayed updated output image.

16. The image inspection computing device in accordance with claim 9 , wherein to annotate at least one segment of the input image or to annotate at least one object in the object mask, said processor is configured to annotate the input image or the object mask, based on user input, on a graphical user interface displayed o

Assignees

• Molecular Devices Llc

Inventors

• Al-Kofahi Yousef
• Macdonald Michael
• Singanamalli Asha
• Yousefhussien Mohammed
• Marshall Will

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06V10/82 Primary

  using neural networks · CPC title

• G06F18/2411

  based on the proximity to a decision surface, e.g. support vector machines · CPC title

• G06F18/2413

  based on distances to training or reference patterns · CPC title

• G06T7/11

  Region-based segmentation · CPC title