Drawer vision system

Patent metadata

Field	Value
Publication number	US-10140705-B2
Application number	US-201515317497-A
Country	US
Kind code	B2
Filing date	Jun 10, 2015
Priority date	Jun 10, 2014
Publication date	Nov 27, 2018
Grant date	Nov 27, 2018

Abstract

Methods and systems for detecting properties of sample tubes in a laboratory environment include a drawer vision system that can be trained and calibrated. Images of a tube tray captured by at least one camera are analyzed to extract image patches that allow a processor to automatically determine if a tube slot is occupied, if the tube has a cap, and if the tube has a tube top cup. The processor can be trained using a random forest technique and a plurality of training image patches. Cameras can be calibrated using a three-dimensional calibration target that can be inserted into the drawer.

First claim

What is claimed is:

1. A method for detecting properties of sample tubes, comprising steps of:
a) capturing a series of images of a sample tray using at least one overhead camera;
b) receiving at a processor the series of images of the tray from the at least one overhead camera;
c) extracting, by the processor, a plurality of image patches from each image, each image patch corresponding to a portion of each image based on features in each image;
d) automatically determining, using the processor, from a first subset of the plurality image patches, each patch corresponding to one of a plurality of slots in the tray, whether each of a plurality of slots contains a sample tube;
e) for those plurality of slots that contain a sample tube, automatically determining, using the processor, from a second subset of the plurality image patches, each patch corresponding to the top of the sample tube, whether each sample tube has a cap; and
f) for those tubes that do not have a cap, automatically determining, using the processor, from the second subset of the plurality image patches whether each sample tube has a tube-top cup or is a plain tube.

2. The method of claim 1 , wherein the series of images comprises images of the tray at predetermined positions in a tray drawer.

3. The method of claim 1 , wherein the processor uses a set of fiducial markers on the tray surface to determine the location of each patch corresponding to one of a plurality of slots in the tray.

4. The method of claim 1 , wherein the step of automatically determining whether each of a plurality of slots contains a sample tube comprises:
a. for each slot, identifying a patch in at least one image in the series of images that corresponds to that slot based on optical marks on the tray surface; and
b. determining, for each identified patch, a probability that the slot is occupied by a sample tube.

5. The method of claim 1 , wherein the step of automatically determining whether each sample tube has a cap comprises:
a. for each sample tube, identifying a patch in at least one image in the series of images that corresponds to the top of the sample tube based on the detection of a circle in the at least one image; and
b. determining, for each identified patch, a probability that the sample tube has a cap.

6. The method of claim 1 , wherein the step of automatically determining whether each sample tube has a tube-top cup comprises:
a. for each sample tube, identifying a patch in at least one image in the series of images that corresponds to the top of the sample tube; and
b. determining, for each identified patch, a probability that the sample tube has a tube-top cup.

7. The method of claim 1 , further comprising the step of automatically determining, using the processor, for each sample tube, at least one of: tube type; tube height, tube diameter; tube offset; cap color; and fluid type.

8. The method of claim 1 , further comprising the step of automatically identifying, using the processor, a tray type from the series of images.

9. The method of claim 1 , further comprising the step of calibrating at least one camera, which is configured to capture the plurality of images, using a 3D target having a plurality of unique digital markers.

10. The method of claim 1 , further comprising the step of training the processor to perform the determining steps using a random forest technique and a plurality of training images.

11. A vision system for use in an in vitro diagnostics environments comprising:
a drawer configured to receive a tray, wherein the tray comprises a plurality of slots, each configured to receive a sample tube;
at least one overhead camera configured to capture a series of images of the tray as the drawer is moved;
a processor configured to perform the following steps:
a. receiving the series of images of the tray from the at least one camera;
b. extracting a plurality of image patches from each image, each image patch corresponding to a portion of each image based on features in each image;
c. automatically determining, from a first subset of the plurality image patches, each patch corresponding to one of a plurality of slots in the tray, whether each of a plurality of slots contains a sample tube;
d. for those plurality of slots that contain a sample tube, automatically determining, from a second subset of the plurality image patches, each patch corresponding to the top of the sample tube, whether each sample tube has a cap; and
e. for those tubes that do not have a cap, automatically determining, from the second subset of the plurality image patches whether each sample tube has a tube-top cup.

12. The system of claim 11 , wherein the series of images comprises images of the tray at predetermined positions in the drawer.

13. The system of claim 11 , wherein a set of fiducial markers on the surface of the tray to determine each patch corresponding to one of a plurality of slots in the tray.

14. The system of claim 11 , wherein the step of automatically determining whether each of a plurality of slots contains a sample tube comprises:
a. for each slot, identifying a patch in at least one image in the series of images, which corresponds to that slot based on optical marks on the tray surface; and
b. determining, for each identified patch, a probability that the slot is occupied by a sample tube.

15. The system of claim 11 , wherein the step of automatically determining whether each sample tube has a cap comprises:
a. for each sample tube, identifying a patch in at least one image in the series of images that corresponds to the top of the sample tube based on the detection of a circle in the at least one image; and
b. determining, for each identified patch, a probability that the sample tube has a cap.

16. The system of claim 11 , wherein the step of automatically determining whether each sample tube has a tube-top cup comprises:
a. for each sample tube, identifying a patch in at least one image in the series of images that corresponds to the top of the sample tube; and
b. determining, for each identified patch, a probability that the sample tube has a tube-top cup.

17. The system of claim 11 , further comprising the step of automatically determining, using the processor, for each sample tube, at least one of: tube type; tube height; tube diameter; tube offset; cap color; and fluid type.

18. The system of claim 11 , wherein the processor is configured to perform the step of automatically identifying a tray type from the series of images.

19. The system of claim 11 , wherein the processor is configured to perform the step of calibrating the at least one camera using a 3D target having a plurality of unique digital markers.

20. The system of claim 11 , wherein the processor is configured to perform the step of training the processor to perform the determining steps using a random forest technique and a plurality of training images.

Assignees

• Siemens Healthcare Diagnostics Inc

Inventors

• Wu Wen
• Chang Yao-Jen
• Liu David
• Pollack Benjamin
• Chen Terrence

Classifications

• G06V20/52

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/24323

  Tree-organised classifiers · CPC title

• G01C11/06

  by comparison of two or more pictures of the same area · CPC title

• G01N21/952

  Inspecting the exterior surface of cylindrical bodies or wires (G01N21/956 takes precedence) · CPC title