Method and system for automatically mapping fluid objects on a substrate

Patent metadata

Field	Value
Publication number	US-2020124837-A1
Application number	US-201916583925-A
Country	US
Kind code	A1
Filing date	Sep 26, 2019
Priority date	Oct 19, 2018
Publication date	Apr 23, 2020
Grant date	—

Abstract

A method and system for mapping fluid objects on a substrate using a microscope inspection system that includes a light source, imaging device, stage for moving a substrate disposed on the stage, and a control module. A computer analysis system includes an object identification module that identifies for each of the objects on the substrate, an object position on the substrate including a set of X, Y, and θ coordinates using algorithms, networks, machines and systems including artificial intelligence and image processing algorithms. At least one of the objects is fluid and has shifted from a prior position or deformed from a prior size.

First claim

1 . A method for mapping fluid objects on a substrate using a microscope inspection system comprising:
providing a microscope inspection system comprising a microscope system and a computer analysis system,
wherein the microscope system includes a light source, imaging device, stage for moving a substrate disposed on the stage, and a control module, and
wherein the computer analysis system includes an object identification module; and
performing a scan of the substrate using the microscope inspection system; identifying for each of the objects on the substrate, an object position on the substrate including a set of X, Y, and θ coordinates using algorithms, networks, machines and systems including artificial intelligence and image processing algorithms,
wherein at least one of the objects is fluid and has shifted from a prior position or deformed from a prior size; and
generating object mapping information that reflects the position of each of the objects and a shift or deformity amount for each of the objects,
wherein the step of generating object mapping information is done automatically using algorithms, networks, machines and systems including artificial intelligence and image processing algorithms.

2 . A method according to claim 1 , further comprising aligning the stage, the imaging device and objects to reflect the position of the at least one object that has shifted.

3 . A method according to claim 1 , wherein generating object mapping information comprises generating an object layout map.

4 . A method according to claim 1 , wherein the object includes electronic objects on a substrate or biological objects such as cells, tissue or the like found in a biological specimen mounted on a slide.

5 . A method according to claim 1 , wherein the computer analysis system further includes an object layout prediction module.

6 . A method according to claim 1 , wherein the microscope system further includes a low resolution objective and a high resolution objective.

7 . A method according to claim 1 , wherein the object identification module is configured to receive a low or high resolution scan of the substrate from the microscope system.

8 . A method according to claim 7 , wherein the object identification module is configured to detect one or more objects in the received low or high resolution scan, using computer vision, one or more artificial intelligence algorithms and/or computer algorithms.

9 . A method according to claim 7 , wherein the object identification module can apply an image processing algorithm to the received substrate scan and for each object on the substrate: i) detect the object; ii) determine an object type; iii) determine orientation; iv) identify an imaging alignment position; and/or v) identify a starting scan position, and
wherein the object identification module can use the object mapping information in connection with a reference point on the substrate to determine the X, Y, θ coordinates of: i) each object on the substrate: ii) an imaging alignment position for each object on a substrate; and/or iii) a starting scan position for each object on the substrate.

10 . A method for mapping fluid objects on a substrate using a microscope inspection system comprising:
providing a microscope inspection system comprising a microscope system and a computer analysis system,
wherein the microscope system includes a light source, imaging device, stage for moving a substrate disposed on the stage, and a control module, and
wherein the computer analysis system includes an object identification module and an object layout prediction module; and
performing a scan of the substrate using the microscope inspection system; identifying for each of the objects on the substrate, an object position on the substrate including a set of X, Y, and θ coordinates using algorithms, networks, machines and systems including artificial intelligence and image processing algorithms,
wherein at least one of the objects is fluid and has shifted from a prior position or deformed from a prior size; and
generating object mapping information that reflects the position of each of the objects and a shift or deformity amount for each of the objects,
wherein the step of generating object mapping information is done automatically using algorithms, networks, machines and systems including artificial intelligence and image processing algorithms.

11 . A method according to claim 10 , wherein the microscope system further includes a low resolution objective and a high resolution objective.

12 . A method according to claim 10 , wherein the object identification module is configured to receive a low or high resolution scan of the substrate from the microscope system.

13 . A method according to claim 12 , wherein the object identification module is configured to detect one or more objects in the received low or high resolution scan, using computer vision, one or more artificial intelligence algorithms and/or computer algorithms.

14 . A method according to claim 12 , wherein the object identification module can apply an image processing algorithm to the received substrate scan and for each object on the substrate: i) detect the object; ii) determine an object type; iii) determine orientation; iv) identify an imaging alignment position; and/or v) identify a starting scan position, and
wherein the object identification module can use the object mapping information in connection with a reference point on the substrate to determine the X, Y, θ coordinates of: i) each object on the substrate: ii) an imaging alignment position for each object on a substrate; and/or iii) a starting scan position for each object on the substrate.

15 . A method according to claim 10 , wherein the object prediction layout module receives feedback data and/or the object mapping information from the object identification module.

16 . A method according to claim 15 , wherein the feedback data includes X, Y, θ coordinates for each object on the substrate at a specific stage in a manufacturing or examination process, and the amount each object on the substrate has deformed, shifted and/or changed its orientation during the manufacturing or examination process.

17 . A method according to claim 16 , wherein the object layout prediction module can receive an initial object layout of the substrate and apply a layout prediction algorithm using artificial intelligence to determine a new object layout of the substrate at a particular stage in a manufacturing and/or examination process.

18 . A method according to claim 10 , wherein the object prediction layout module is implemented using a linear regression model or a multiple linear regression model.

19 . A method according to claim 10 , wherein the object prediction layout module generates an alert when the X, Y, θ coordinates of the object exceed a predefined tolerance for a type of object and/or substrate being inspected.

20 . A system for mapping fluid objects on a substrate comprising:
a microscope inspection system including a microscope system,
wherein the microscope system includes a light source, imaging device, stage for moving a substrate disposed on the stage, and a control module, and
wherein the imaging device scans the substrate;
an object layout identification module for identifying for each of the objects on the substrate, an object position on the substrate including a set of X, Y, and θ coordinates using algorithms, networks, machines and systems including artifici

Assignees

• Nanotronics Imaging Inc

Inventors

• Putman Matthew C
• Putman John B
• Cruickshank John
• Orlando Julie
• Frankel Adele
• Scott Brandon

Classifications

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2207/30024

  Cell structures in vitro; Tissue sections in vitro · CPC title

• G06T7/73

  using feature-based methods · CPC title

• G06T7/001

  using an image reference approach · CPC title

• G02B21/0032

  Optical details of illumination, e.g. light-sources, pinholes, beam splitters, slits, fibers (G02B21/0036 - G02B21/008; means for illumination of specimens in general G02B21/06) · CPC title