Methods and systems for learning-based image edge enhancement of sample tube top circles

Patent metadata

Field	Value
Publication number	US-11600058-B2
Application number	US-201816630274-A
Country	US
Kind code	B2
Filing date	Jun 25, 2018
Priority date	Jul 11, 2017
Publication date	Mar 7, 2023
Grant date	Mar 7, 2023

Abstract

Methods for image-based detection of the tops of sample tubes used in an automated diagnostic analysis system may be based on a convolutional neural network to pre-process images of the sample tube tops to intensify the tube top circle edges while suppressing the edge response from other objects that may appear in the image. Edge maps generated by the methods may be used for various image-based sample tube analyses, categorizations, and/or characterizations of the sample tubes to control a robot in relationship to the sample tubes. Image processing and control apparatus configured to carry out the methods are also described, as are other aspects.

First claim

What is claimed is:

1. An image processing and control apparatus, comprising:
image capture apparatus configured to capture an image of one or more tops of one or more respective sample tubes;
a robot configured to move the one or more respective sample tubes; and
a system controller comprising a processor and a memory, the system controller configured via programming instructions stored in the memory to process the image of the one or more tops of the one or more respective sample tubes by applying the image to a convolutional neural network to:
identify and intensify sample tube top edges appearing in the image;
identify and suppress edge responses from other objects appearing in the image;
generate an edge map of the image of the one or more tops of the one or more respective sample tubes; and
control the robot to move one or more sample tubes based on the generated edge map;
wherein the convolutional neural network is a patch-based convolutional network comprising a plurality of convolution layers followed by a fusion module that fuses edge responses of individual patches into one edge map representing an input image.

2. The image processing and control apparatus of claim 1 , wherein the robot comprises an end effector configured to grasp and move the one or more sample tubes based on the edge map.

3. The image processing and control apparatus of claim 1 , further comprising a tube tray configured to receive the one or more sample tubes therein.

4. An image processing and control apparatus, comprising:
image capture apparatus configured to capture an image of one or more tops of one or more respective sample tubes;
a robot configured to move the one or more respective sample tubes; and
a system controller comprising a processor and a memory, the system controller configured via programming instructions stored in the memory to process the image of the one or more tops of the one or more respective sample tubes by applying the image to a convolutional neural network to:
identify and intensify sample tube top edges appearing in the image;
identify and suppress edge responses from other objects appearing in the image;
generate an edge map of the image of the one or more tops of the one or more respective sample tubes; and
control the robot to move one or more sample tubes based on the generated edge map; wherein:
the convolutional neural network is a fully convolutional network comprising a plurality of convolution layers;
the fully convolutional network comprises one or more convolution layers and one or more max-pooling layers followed by one or more fully-connected convolutional layers;
a first convolution layer includes a kernel size of 5, a stride of 1, and a pad of 0;
a second convolution layer includes a kernel size of 5, a stride of 1, and a pad of 0;
a first fully-connected convolution layer includes a kernel size of 5, a stride of 1, and a pad of 0; and
a second fully-connected convolution layer includes a kernel size of 1, a stride of 1, and a pad of 0.

5. The image processing and control apparatus of claim 1 , wherein the patch-based convolutional network comprises one or more convolution layers and one or more max-pooling layers followed by one or more fully-connected layers.

6. The A non-transitory computer-readable medium comprising computer instructions of a fully convolutional network and parameters thereof capable of being executed in a processor and of applying the fully convolutional network and the parameters to an image of one or more sample tube tops to identify and intensify sample tube top edges appearing in the image and generate an edge map to be stored in the non-transitory computer-readable medium and accessible to a controller to control a robot based on the edge map, wherein the fully convolutional network comprises a first convolution layer that includes a kernel size of 5, a stride of 1, and a pad of 0, and further comprises a first max-pooling layer that includes a kernel size of 2, a stride of 2, and a pad of 0.

7. A method of processing an image of sample tube tops and controlling a robot based thereon, comprising:
receiving an input image of one or more tops of one or more respective sample tubes;
applying to the input image a fully convolutional network having one or more convolution layers and one or more max-pooling layers;
identifying and intensifying sample tube top edges appearing in the image in response to the applying;
generating an edge map in response to the applying and the intensifying;
determining sample tube categories or characteristics based on the generated edge map; and
controlling a robot based on the determined sample tube categories or characteristics;
wherein a first convolution layer includes a kernel size of 5, a stride of 1, and a pad of 0.

8. A method of processing an image of sample tube tops and controlling a robot based thereon, comprising:
receiving an input image of one or more tops of one or more respective sample tubes;
applying to the input image a fully convolutional network having one or more convolution layers and one or more max-pooling layers;
identifying and intensifying sample tube top edges appearing in the image in response to the applying;
generating an edge map in response to the applying and the intensifying;
determining sample tube categories or characteristics based on the generated edge map; and
controlling a robot based on the determined sample tube categories or characteristics;
wherein a first max-pooling layer includes a kernel size of 2, a stride of 2, and a pad of 0.

Assignees

• Siemens Healthcare Diagnostics Inc

Inventors

• Chang Yao-Jen
• Kluckner Stefan
• Pollack Benjamin S
• Chen Terrence

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/08

  Learning methods · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06V10/454 Primary

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06T2207/30164

  Workpiece; Machine component · CPC title