Sample image generation device, sample image generation method, sample image generation system, and recording medium

Patent metadata

Field	Value
Publication number	US-2024428554-A1
Application number	US-202418826399-A
Country	US
Kind code	A1
Filing date	Sep 6, 2024
Priority date	Mar 17, 2022
Publication date	Dec 26, 2024
Grant date	—

Abstract

A sample image generation device includes a memory and a processor. A first image is an image obtained by capturing an image of a sample. A predetermined direction in the first image is a direction in which a virtual observation optical system is present among optical axis directions of the virtual observation optical system. The processor acquires the first image from the memory, divides the first image into a plurality of areas, acquires a refractive index distribution of the sample from the memory, calculates a point spread function using the refractive index distribution, generates second images using the point spread function, combines the second images, and generates a third image corresponding to the first image. In the calculation process, the point spread function of a first area is calculated using the refractive index distribution of each area included in an area group. The first area is an area for which the point spread function is to be calculated. The area group is constituted of a plurality of areas inside a range in which light rays originating from the first area radiate in the predetermined direction.

First claim

What is claimed is:

1 . A sample image generation device comprising a memory and a processor, wherein
a first image is an image obtained by capturing an image of a sample, a predetermined direction in the first image is a direction in which a virtual observation optical system is present among optical axis directions of the virtual observation optical system, the processor
performs a first acquisition process of acquiring the first image from the memory,
performs a division process of dividing the acquired first image into a plurality of areas,
performs a second acquisition process of acquiring a refractive index distribution of the sample from the memory,
performs a calculation process of calculating respective point spread functions for the divided areas, using the acquired refractive index distribution,
performs a first generation process of generating respective second images corresponding to the areas, using the respective point spread functions calculated for the areas, and
combines the respective second images corresponding to the areas and generates a third image corresponding to the first image,
in the calculation process, the point spread function of a first area is calculated using a refractive index distribution of each of areas included in an area group, the first area is an area for which the point spread function is to be calculated, and the area group is constituted of a plurality of areas inside a range in which light rays originating from the first area radiate in the predetermined direction, and includes an area outside a range defined by extending the first area in the predetermined direction.

2 . The sample image generation device according to claim 1 , wherein in the calculation process, the processor sets a point light source in the first area and calculates a point spread function of the first area using a first wavefront from a wave source of which is the set point light source.

3 . The sample image generation device according to claim 2 , wherein
in the calculation process, the processor calculates a second wavefront, using the first wavefront and the refractive index distribution corresponding to each of the areas included in the area group, the second wavefront being a wavefront propagating through the sample in the predetermined direction, calculates an intensity distribution corresponding to a third wavefront, using the calculated second wavefront, the third wavefront being a wavefront at a position of a focal plane of the virtual observation optical system, and calculates a point spread function of the first area, using the calculated intensity distribution.

4 . The sample image generation device according to claim 3 , wherein
in the calculation process, the processor determines whether a wavefront propagating through the sample has reached an outer edge of the sample, in the predetermined direction, and the second wavefront is a wavefront at a position where the wavefront is determined to have reached the outer edge.

5 . The sample image generation device according to claim 3 , wherein the second wavefront is a wavefront after passing through the sample and before reaching the virtual observation optical system.

6 . The sample image generation device according to claim 1 , wherein
in the second acquisition process, the processor acquires the refractive index distribution for each of small areas obtained by further dividing the divided area, and in the calculation process, the processor
calculates point spread functions of the small areas, using the refractive index distribution of the small areas, and
calculates a point spread function of the area, using the point spread functions of the small areas.

7 . The sample image generation device according to claim 1 , wherein
the processor
performs a provisional calculation process of provisionally calculating respective point spread functions for the areas divided in the division process, and
performs the division process such that the area in which an intensity peak value of the provisionally calculated point spread function is less than ⅕ of a reference is set to be smaller than the area in which an intensity peak value of the provisionally calculated point spread function is equal to or greater than ⅕ of the reference, and
the reference is an intensity peak value of a point spread function when the sample is not present.

8 . The sample image generation device according to claim 1 , wherein
the processor performs an estimation process of estimating the sample, and in the division process, in a direction orthogonal to the predetermined direction, a size of an area that is an area of the estimated sample and to which an outer edge of the estimated sample does not belong is set to be smaller than a size of an area that is not an area of the estimated sample and to which the outer edge of the estimated sample does not belong.

9 . The sample image generation device according to claim 3 , wherein
in the calculation process, the processor
calculates an excitation light intensity at a position of the set point light source,
calculates a fluorescence intensity distribution, using the calculated intensity distribution and the calculated excitation light intensity, and
calculates a point spread function of the first area, using the calculated fluorescence intensity distribution.

10 . The sample image generation device according to claim 9 , wherein in the calculation process, the processor calculates the excitation light intensity, using a refractive index distribution of an excitation light wavelength.

11 . The sample image generation device according to claim 9 , wherein
the first image is an image obtained by capturing an image with a device with a confocal pinhole, and in the calculation process, the processor
calculates an intensity of light passing through the confocal pinhole, using the calculated intensity distribution,
calculates a fluorescence intensity, using the calculated excitation light intensity and the calculated intensity of light, and
calculates a point spread function of the first area, using the calculated fluorescence intensity.

12 . The sample image generation device according to claim 3 , wherein in the calculation process, the processor calculates the second wavefront using a beam propagation method.

13 . A sample image generation system comprising:
an observation optical system configured to form an optical image of a sample; an imager configured to capture the optical image; and the sample image generation device according to claim 1 .

14 . A sample image generation system comprising a memory and a processor, wherein
a first image is an image obtained by capturing an image of a sample, a predetermined direction in the first image is a direction in which a virtual observation optical system is present among optical axis directions of the virtual observation optical system, the processor
performs a first acquisition process of acquiring the first image from the memory,
performs a division process of dividing the acquired first image into a plurality of areas,
performs a second acquisition process of acquiring a refractive index distribution of the sample from the memory,
performs a calculation process of calculating respective point spread functions for the divided areas, using the acquired refractive index distribution,
performs a first generation process of generating respective second images corresponding to the areas, using the respective point spread functions calculated for the areas, and
combines the respective second images corre

Assignees

• Evident Corp

Inventors

• Suzuki Yoshimasa
• Fujii Shintaro

Classifications

• G06T2207/10056

  Microscopic image · CPC title

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title

• G06T7/11

  Region-based segmentation · CPC title

• G06T5/73

  Deblurring; Sharpening · CPC title

• G02B21/008

  Details of detection or image processing, including general computer control · CPC title