Locality-based detection of tray slot types and tube types in a vision system

Patent metadata

Field	Value
Publication number	US-10319092-B2
Application number	US-201615551571-A
Country	US
Kind code	B2
Filing date	Feb 16, 2016
Priority date	Feb 18, 2015
Publication date	Jun 11, 2019
Grant date	Jun 11, 2019

Abstract

A method for detecting properties of sample tubes is provided that includes extracting image patches substantially centered on a tube slot of a tray or a tube top in a slot. For each image patch, the method may include assigning a first location group defining whether the image patch is an image center, a corner of an image or a middle edge of an image, selecting a trained classifier based on the first location group and determining whether each tube slot contains a tube. The method may also include assigning a second location group defining whether the image patch is from an image center, a left corner of the image, a right corner of the image, a left middle of the image; a center middle of the image or a right middle of the image, selecting a trained classifier based on the second location group and determining a tube property.

First claim

We claim:

1. A method for detecting properties of sample tubes, comprising steps of:
receiving a series of images of a tray acquired by one or more cameras, the tray having a plurality of tube slots;
extracting, using a processor, a plurality of image patches from each image, wherein each of the plurality of image patches are substantially centered on one of a tube slot and a tube top;
assigning, to each image patch, a first location group that defines whether the image patch is from one of: a center of the image, a corner of the image, and a middle edge of the image;
selecting, for each image patch, based on the first location group, a trained classifier to use in processing the image patch;
automatically determining, using the processor, from the plurality image patches, whether each tube slot in the tray contains a tube using the trained classifier for each image patch.

2. The method of claim 1 , wherein the tray is configured to fit within a portion of a drawer movable between an open position and a closed position and the series of images of the tray are acquired via the one or more cameras as the drawer is moved between the open and the closed position.

3. The method of claim 1 , further comprising:
assigning, to each image patch, a second location group that defines whether the image patch is from one of: the center of the image, a left corner of the image, a right corner of the image, a left middle of the image; a center middle of the image and a right middle of the image; and
selecting, for each image patch, based on the second location group, the trained classifier to use in processing the image patch,
wherein, when it is determined that one or more of the tube slots contains a tube, the method further comprises automatically determining, using the processor, from the plurality image patches, at least one property of each of the tubes contained in the one or more tube slots.

4. The method of claim 3 , wherein determining at least one property of each of the tubes further comprises automatically determining, using the processor, from the plurality image patches, whether each of the tubes contained in the one or more tube slots has a cap based on the corresponding trained classifier.

5. The method of claim 3 , wherein determining at least one property of each of the tubes further comprises automatically determining, using the processor, from the plurality image patches, whether each tube contained in the one or more tube slots has a tube-top sample cup or is a plain tube based on the corresponding trained classifier.

6. The method of claim 3 , wherein
receiving the series of images further comprises receiving the series of images from a first camera and a second camera adjacent to the first camera,
extracting the plurality of image patches further comprises extracting image patches from each image received from the first camera and extracting image patches from each image received from the second camera,
assigning the second location group further comprises assigning the second location group to each image patch extracted from images received from the first camera horizontally symmetric to each image patch extracted from images received from the second camera, and
selecting the trained classifier further comprises selecting the same trained classifier for each image patch extracted from images received from the first camera that is horizontally symmetric to each image patch extracted from images received from the second camera.

7. The method of claim 6 , wherein the left corner of the image, the right corner of the image, and the center middle of the image each comprise a plurality of image patches, and
assigning the second location group horizontally symmetrical further comprises:
using a row of image patches from of one of the first camera and the second camera as a reference location; and
aligning image patches from the other of the first camera and the second camera to the reference location.

8. The method of claim 1 , wherein each image comprises a matrix of three rows of tube slots and three columns of tube slots and the plurality of image patches comprise a matrix of three rows of image patches and three columns of image patches, each image patch corresponding to a location of one of the tube slots in the image.

9. A method for offline image patch classifier training, comprising steps of:
receiving a series of images of a tray from a plurality of cameras, the tray having a plurality of tube slots;
extracting a plurality of image patches from each image, wherein each of the plurality of image patches are substantially centered on one of a tube slot and a tube top;
providing, using a processor, each image patch of the plurality of images to a classifier;
collecting, using the processor, image patch data for each image patch provided to the classifier, the data indicating one of: whether each tube slot in the tray contains a tube; whether each of the tubes contained in the one or more tube slots has a cap; and whether each tube contained in the one or more tube slots has a tube-top sample cup or is a plain tube; and
determining, using the processor, image patch classifiers corresponding to each image patch based on the image patch data.

10. The method of claim 9 , wherein extracting the plurality of image patches from each image further comprising extracting, over time, multiple image patches substantially centered on one of the same tube slot and the same tube top.

11. The method of claim 9 , wherein the classifier is a random forest classifier, a support vector machine classifier, or a probabilistic boosting tree classifier.

12. A vision system for use in an in vitro diagnostics environments comprising:
a tray comprising a plurality of slots arranged in a matrix of rows and columns, each tube slot configured to receive a sample tube;
a surface configured to receive the tray;
an image capture system having a first camera configured to capture a series of images of the tray; and
a processor configured to:
receive the series of images of the tray captured by the first camera;
extract a plurality of image patches from each image of the series of images, wherein each of the plurality of image patches are substantially centered on one of the plurality of tube slots or a tube top;
assign, to each image patch, a first location group that defines whether the image patch is from one of: the center of the image, a corner of the image, and a middle edge of the image;
select, for each image patch, based on the first location group, a trained classifier to use in processing the image patch; and
automatically determine, from the plurality of image patches, whether each tube slot in the tray contains a corresponding sample tube using the trained classifier for each image patch.

13. The system of claim 12 , wherein the image capture system further comprises a second camera adjacent to the first camera and configured to capture images of the tray proximate to the images captured by the first camera.

14. The system of claim 12 , wherein the surface comprises a portion of a drawer movable between an open and a closed position and the image of the tray is captured via the first camera and the second camera as the drawer is moved between the open position and the closed position.

15. The system of claim 13 , wherein the processor is further configured to:
extract image patches from each image received from the first camera and extract image patches from each image received from the second camera;
assign the second location group to each image patch extracted from images received from the f

Assignees

• Siemens Healthcare Diagnostics Inc

Inventors

• Wu Wen
• Pollack Benjamin
• Chang Yao-Jen
• Dumont Guillaume
• Chen Terrence

Classifications

• G01N2035/0091

  GUI [graphical user interfaces] · CPC title

• G01N35/00732

  Identification of carriers, materials or components in automatic analysers · CPC title

• G01N35/00871

  Communications between instruments or with remote terminals · CPC title

• G01N2035/00881

  network configurations · CPC title

• G06T2207/30108

  Industrial image inspection · CPC title