Systems and methods for detection, analysis, isolation and/or harvesting of biological objects

Patent metadata

Field	Value
Publication number	US-10564172-B2
Application number	US-201515309712-A
Country	US
Kind code	B2
Filing date	May 8, 2015
Priority date	May 8, 2014
Publication date	Feb 18, 2020
Grant date	Feb 18, 2020

Abstract

Systems and methods provide for detection and controlled interaction with one or more objects. The system can include an imaging subsystem (20), a tool subsystem (26) containing one or more tools, a stage subsystem (16) and a control system (40). The control system (40) can integrate controls for each of the other subsystems, which controls can be implement desired functions over a variety of process parameters to perform the controlled interaction.

First claim

What is claimed is:

1. A system comprising:
a stage, the stage being moveable by a stage motion system along at least two orthogonal axes of the stage;
an imaging device supported relative to the stage and spaced apart from a tool;
a tool holder supporting the tool above a surface of the stage, the tool including a body extending toward the stage and terminating in a tip thereof, wherein the tip of the tool is configured to interact with the surface of the stage that is outside a field of view (FOV) of the imaging device;
a tool motion system configured to position the tool relative to the stage;
memory to store stage position data for the stage in the at least two orthogonal axes of the stage; and
a control system configured to:
position the stage at a first position and cause the tip of the tool to interact with the surface of the stage to provide a visible mark on the surface of the stage that is outside the FOV of the imaging device;
identify first spatial coordinates for the visible mark in the at least two orthogonal axes of the stage based on the stage position data representing the first position;
position the stage at a second position to position the visible mark in the FOV;
cause the imaging device to capture at least one image for the FOV in response to positioning the visible mark within the FOV of the imaging device, wherein the at least one image includes the visible mark;
identify an optical location associated with at least one pixel in the FOV based on the at least one image;
identify second spatial coordinates for the visible mark with respect to the at least two orthogonal axes of the stage based on the optical location and the stage position data representing the second position, wherein the optical location is associated with respective spatial coordinates in the at least two orthogonal axes of the stage corresponding to the second spatial coordinates;
determine a spatial offset between the first spatial coordinates and the second spatial coordinates corresponding to a tip-to-optical offset between the tip of the tool with respect to the optical location within the FOV of the imaging device;
position the stage relative to the tip based on the spatial offset to enable the tip to interact with a target location identified based on another at least one image;
repeat each of the following: the controlling of the tool motion system to position the tip of the tool relative to the stage to contact the surface of the stage, the identifying of the first spatial coordinates for the visible mark, and the moving of the of the stage relative to the imaging device for a plurality of different touchoff locations, wherein each of the different touchoff locations includes the visible mark;
determine a spatial offset between each touchoff location of the plurality of different touchoff locations and the optical location associated with the at least one pixel in the FOV based on at least one respective image; and
determine an average spatial offset based on the determined spatial offsets between each touchoff location of the plurality of different touchoff locations and the optical location associated with the at least one pixel in the FOV, wherein the control system is to position the stage relative to the tip based on the average spatial offset.

2. The system of claim 1 , wherein the control system is configured to:
control the tool motion system to position the tip of the tool relative to the stage to contact the surface of the stage to provide the visible mark on the surface of the stage in response to the stage being positioned at the first position outside the FOV of the imaging device;
identify the first spatial coordinates for the visible mark with respect to the at least two orthogonal axes of the stage based on the stage position data;
move the stage relative to the imaging device to position the stage at the second position and align the visible mark with the optical location associated with the at least one pixel in the FOV; and
identify the second spatial coordinates for the visible mark with respect to the at least two orthogonal axes of the stage in response to aligning the visible mark with the optical location associated with the at least one pixel in the FOV based on the stage position data.

3. The system of claim 2 , wherein the control system is further configured to control the tool motion system to move the tip along a tool axis that is orthogonal with respect to the surface of the stage to provide the visible mark on the surface of the stage in response to the tool being positioned at a reference location with respect to the at least two orthogonal axes of the stage.

4. The system of claim 1 , further comprising:
a tip position sensor fixed with respect to the surface of the stage and configured to provide tip position data representing a tip location of the tip with respect to each of the at least two orthogonal axes of the stage.

5. The system of claim 4 , wherein the tool motion system is configured to position the tool relative to the stage at a predefined location, wherein the control system is configured to determine the tip location for the tip with respect to each of the at least two orthogonal axes of the stage based on the tip position data provided by the tip position sensor in response to the tool being positioned at the predefined location.

6. The system of claim 1 , wherein the at least one pixel in the FOV of the imaging device comprises one of a center pixel located in a center of the FOV or a predetermined pixel at a periphery of the FOV.

7. The system of claim 1 , wherein the control system associates metadata with each captured image, the metadata associated with each captured image comprising a resolution for the respective captured image and spatial coordinates for the stage for each respective image that is captured.

8. The system of claim 7 , wherein the at least one image for the FOV comprises a plurality of images for different spatial regions of the surface of the stage aggregated into a montage image, each of the plurality of images being associated with corresponding metadata.

9. The system of claim 1 ,
further comprising a plurality of tools independently moveable relative to the stage,
wherein the tool holder is configured to support the plurality of tools and each of the plurality of tools includes a respective tip, the control system being configured for each tool of the plurality of tools:
position the stage at a given position and cause each tip of the tool to interact with the surface of the stage to provide a respective visible mark on the surface of the stage that is outside the FOV of the imaging device;
identify spatial coordinates for the respective visible mark in the at least two orthogonal axes of the stage based on the stage position data;
position the stage at another position to position the respective visible mark in the FOV of the imaging device,
cause the imaging device to capture at least one respective image for the FOV in response to positioning the respective visible mark within the FOV of the imaging device;
identify a respective optical location associated with the at least one pixel in the FOV based on the least one respective image;
identify spatial coordinates for the respective visible mark with respect to the at least two orthogonal axes of the stage based on the respective optical location, wherein the respective optical location is associated with corresponding spatial coordinates in the at least two orthogonal axes of the stage corresponding to the spatial coordinates identified for the respective visible mark; and
determine a respective spatial offset between the spatial coordinates identified for the respective visibl

Assignees

• Cleveland Clinic Found
• Parker Hannifin Corp

Inventors

• Muschler George F
• Monnich James K
• Kwee Edward J
• Powell Kimerly A
• Herderick Edward E
• Boehm Cynthia A
• Adams Thomas R
• Germanoski Robert
• Krakosh Iii Frank
• Dunn James
• Bantz Daniel

Classifications

• B01L9/56

  Means for indicating position of a recipient or sample in an array · CPC title

• G01B11/14

  for measuring distance or clearance between spaced objects or spaced apertures (G01B11/26 takes precedence; rangefinders G01C3/00) · CPC title

• G01N2035/1013

  Confirming presence of tip · CPC title

• B01L2300/0654

  Lenses; Optical fibres · CPC title

• G01B11/0608

  Height gauges · CPC title