Image-based tray alignment and tube slot localization in a vision system

Patent metadata

Field	Value
Publication number	US-10725060-B2
Application number	US-201615551569-A
Country	US
Kind code	B2
Filing date	Feb 16, 2016
Priority date	Feb 18, 2015
Publication date	Jul 28, 2020
Grant date	Jul 28, 2020

Abstract

A method of tube slot localization is provided using a tray coordinate system and a camera coordinate system. The method includes receiving, a series of images from at least one camera of a tray comprising tube slots arranged in a matrix of rows and columns. Each tube slot is configured to receive a sample tube. The method also includes automatically detecting fiducial markers disposed on cross sectional areas between the tube slots on the tray and receiving an encoder value indicating when each row of the tray is substantially at the center of the camera's field of view. The method further includes determining calibration information to provide mapping of locations from the tray coordinate system to locations from the camera coordinate system and automatically aligning the tray based on the encoder value and calibration information.

First claim

We claim:

1. A method of tray calibration using a tray coordinate system having an x-axis and a y-axis and a camera coordinate system having an x-axis and a y-axis, the method comprising:
acquiring an image of a tray, the tray comprising a plurality of tube slots arranged in a matrix of rows and columns, each tube slot configured to receive a sample tube, the image of the tray being acquired via a camera;
detecting, using a processor, a plurality of fiducial markers disposed on cross sectional areas between the tube slots on the tray;
pre-aligning the tray coordinate system with the camera coordinate system;
calculating, using the processor, a tray grid by fitting parallel lines to positions of markers along their x-coordinates of the tray coordinate system and fitting pairs of parallel lines to positions of the markers on each row along their y-coordinates of the tray coordinate system, including performing linear regression to fit the parallel lines to positions of the markers along their x-coordinates of the tray coordinate system and to fit the pairs of parallel lines to positions of the markers on each row along their y-coordinates of the tray coordinate system; and
calibrating, using the processor, the tray by identifying a correspondence of each detected fiducial marker to a physical position on the tray to provide mapping of the tray coordinate system to the camera coordinate system.

2. The method of claim 1 , wherein the tray is configured to fit within a portion of a drawer movable between an open position and a closed position and the image of the tray is acquired via the camera as the drawer is moved between the open and the closed position.

3. The method of claim 1 , further comprising:
acquiring another image of the tray via another camera, the other camera being adjacent the camera and having another camera coordinate system;
pre-aligning the tray coordinate system with the other camera coordinate system; and
identifying the correspondence of each detected fiducial marker to the physical position on the tray to provide mapping of the tray coordinate system to the other camera coordinate system.

4. The method of claim 1 , wherein calculating the tray grid further comprises:
grouping detected markers into clusters along their x-coordinates of the tray coordinate system, wherein the detected markers in each cluster lay in a line in the y-direction of the tray coordinate system which is parallel to the other lines in the y-direction of the tray coordinate system formed by the other clusters.

5. The method of claim 1 , wherein the plurality of fiducial markers are white dots.

6. The method of claim 1 , wherein
the cross sectional areas are near diamond shaped areas, and
fiducial markers on each row of the near diamond shaped area comprise top corner fiducial markers located at the top corner of alternating diamond shaped areas and right corner fiducial markers located at the right corner of alternating diamond shaped areas.

7. The method of claim 1 , further comprising using the correspondence of each detected fiducial marker to the physical position on the tray to align the tray during online operation.

8. A method of tray calibration using a tray coordinate system having an x-axis and a y-axis and a camera coordinate system having an x-axis and a y-axis, the method comprising:
acquiring an image of a tray, the tray comprising a plurality of tube slots arranged in a matrix of rows and columns, each tube slot configured to receive a sample tube, the image of the tray being acquired via a camera;
detecting, using a processor, a plurality of fiducial markers disposed on cross sectional areas between the tube slots on the tray;
pre-aligning the tray coordinate system with the camera coordinate system;
calculating, using the processor, a tray grid by fitting parallel lines to positions of markers along their x-coordinates of the tray coordinate system and fitting pairs of parallel lines to positions of the markers on each row along their y-coordinates of the tray coordinate system;
calibrating, using the processor, the tray by identifying a correspondence of each detected fiducial marker to a physical position on the tray to provide mapping of the tray coordinate system to the camera coordinate system,
wherein calibrating the tray further comprises deriving a pose of the camera by minimizing re-projection errors according to:
R,t =argmin R,t Σi∥p i −ƒ( RP i +t,K c ,d c )∥ 2 ,

where {P i } and {p i } are the 3D and 2D correspondences, ƒ( ) is the 3D to 2D projection function from camera coordinate system to its image plane, K c is the intrinsic calibration matrix containing the focal length and skew of the camera axis and the principal point on the image, d c is its lens distortion vector, and the rotation matrix R and translation vector t are the extrinsic parameters describing the pose of the camera image.

9. A method of tube slot localization using a tray coordinate system and a camera coordinate system, comprising:
receiving, using a processor, a series of images from at least one camera of a tray, the tray comprising tube slots arranged in a matrix of rows and columns, each tube slot configured to receive a sample tube, the series of images of the tray being acquired via the at least one camera;
automatically detecting, using the processor, a plurality of fiducial markers disposed on cross sectional areas between the tube slots on the tray;
receiving, using the processor, an encoder value indicating when each row of the tray is substantially at the center of the camera's field of view;
determining, using the processor, calibration information to provide mapping of locations from the tray coordinate system to locations from the camera coordinate system, wherein the calibration information indicates the tray's type, the tray's orientation and the tray's position; and
automatically aligning, using the processor, the tray based on the encoder value and calibration information, including:
predicting locations of fiducial markers based on the encoder value and the calibration information;
automatically determining an offset between the projected locations of the markers and the locations of the detected marker; and
automatically compensating for the offset to align the tray and tube slots that fall in the at least one camera's field of view.

10. The method of claim 9 , wherein the tray is configured to fit within a portion of a drawer movable between an open and a closed position and the image of the tray is acquired via the at least one camera as the drawer is moved between the open position and the closed position.

11. The method of claim 9 , wherein the calibration information indicates correspondence of each detected fiducial marker to a physical position on the tray.

12. The method of claim 9 , further comprising:
defining tube slot grid points at the center of each of the cross sectional areas; and
projecting compensated tube slot grid points onto one or more of the series of images to locate the tube slots based on the determined offset.

13. The method of claim 9 , further comprising:
extracting data corresponding to each tube slot from one or more of the series of images to determine at least one of: whether a tube occupies one of the tube slots and a tube type.

14. A vision system for use in an in vitro diagnostics environment comprising:
a tray comprising:
a plurality of tube slots arranged in a matrix of rows and columns, each tube slot configured to receive a sample tube;
a plurality of cross sectional areas located between the plurality of tube slots; and
a

Assignees

• Siemens Healthcare Diagnostics Inc

Inventors

• Chang Yao-Jen
• Wissmann Patrick
• Wu Wen
• Dumont Guillaume
• Pollack Benjamin
• Chen Terrence

Classifications

• G01N2035/00772

  mechanical or optical code other than bar code · CPC title

• G01N2035/00851

  process control parameters · CPC title

• G06T7/75

  involving models · CPC title

• G06T2207/30208

  Marker matrix · CPC title

• G01N35/04

  Details of the conveyor system {(G01N35/021 - G01N35/028 take precedence)} · CPC title