System and method for image segmentation

Patent metadata

Field	Value
Publication number	US-10540771-B2
Application number	US-201715710588-A
Country	US
Kind code	B2
Filing date	Sep 20, 2017
Priority date	Mar 20, 2015
Publication date	Jan 21, 2020
Grant date	Jan 21, 2020

Abstract

An image segmentation method is disclosed that allows a user to select image component types, for example tissue types and or background, and have the method of the present invention segment the image according to the user's input utilizing the superpixel image feature data and spatial relationships.

First claim

We claim:

1. An image analysis system for segmenting a digital image of a biological sample, the digital image comprising at least a first image component depicting a first tissue type of the biological sample and a second image component depicting a second tissue type of the biological sample or background, the image analysis system comprising an interface for receiving the digital image and a processor configured for performing the segmentation, the segmentation comprising:
generating a plurality of image channels by applying color deconvolution on the received digital image or by separating RGB color components of the digital image into respective image channels, wherein each of the image channels is a digital image whose pixel intensity values correspond to the color components of the respective channel;
identifying a plurality of superpixels in the received digital image;
for each of the superpixels and for each of the image channels, extracting a feature set, the feature set comprising and/or being derived from pixel intensity values of pixels contained in said superpixel;
receiving at least a first and a second marking provided by a user or provided automatically, the first marking covering one or more first ones of the superpixels, the first marked superpixels representing regions of the first image component, the second marking covering one or more second ones of the superpixels, the second marked superpixels representing regions of the second image component;
for each of the plurality of image channels, comparing the image channel specific feature set of at least one of the first marked superpixels with the image channel specific feature set of at least one of the second marked superpixels for obtaining an image channel specific marker-difference score;
representing the plurality of superpixels as a graph, whereby the center of each of the identified superpixels is represented by a node and whereby the nodes representing centers of adjacent superpixels are connected by a respective edge, wherein the computation of each of the edges comprises:
computing, for each of the image channels, a feature set distance by comparing the image channel specific feature sets of the neighboring superpixels connected by said edge;
multiplying the feature set distance obtained for each of the edges and for each of the image channels with the image channel specific marker-difference score computed for said image channel, thereby computing image-channel specific edge weights;
for each of the edges, aggregating all image-channel specific edge weights and using the aggregated edge weight as the edge weight of the edge;

for each unmarked superpixel of the plurality of superpixels:
computing a first combined distance between said unmarked superpixel and the one or more first marked superpixels by means of a graph traversal algorithm, the first combined distance being a derivative of a feature-set-dependent distance between said unmarked superpixel and the one or more first marked superpixels and of a spatial distance between said unmarked superpixel and the one or more first marked superpixels;
computing a second combined distance between said unmarked superpixel and the one or more second marked superpixels by means of a graph traversal algorithm, the second combined distance being a derivative of a feature-set-dependent distance between said unmarked superpixel and the one or more second marked superpixels and of a spatial distance between said unmarked superpixel and the one or more second marked superpixels;
assigning the unmarked superpixel to the first image component if the first combined distance is smaller than the second combined distance and otherwise associating the unmarked superpixel to the second image component, thereby segmenting the digital image.

2. The image analysis system of claim 1 , the representation of the plurality of superpixels as a graph comprising:
for each of the edges, computing a weight by comparing the feature sets of the neighboring superpixels connected by said edge, whereby the weight negatively correlates with the degree of similarity of the compared feature sets.

3. The image analysis system of claim 2 , whereby each of the computed weights assigned to one of the edges is:
a histogram-distance between the two intensity histograms computed for the two nodes connected by said edge;
a histogram-distance between the two gradient magnitude histograms computed for the two nodes connected by said edge;
a histogram-distance between the two gradient direction histograms computed for the two nodes connected by said edge;
a distance computed as a derivative of one or more of said histogram-distances.

4. The image analysis system of claim 2 ,
wherein the computation of the first combined distance for any of the unmarked superpixels comprises:
computing a first path distance for each of a plurality of first paths, each first path connecting the node representing the center of the unmarked superpixel and a node representing a center of one of the first marked superpixels, each first path distance being computed as the sum of the weights of the edges between neighboring superpixels pairs along a respective one of the first paths; and
using the minimum computed first path distance as the first combined distance calculated for the unmarked superpixel;

wherein the computation of the second combined distance for any of the unmarked superpixels comprises:
computing a second path distance for each of a plurality of second paths, each second path connecting the node representing the center of the unmarked superpixel and a node representing a center of one of the second marked superpixels, each second path distance being computed as the sum of the weights of the edges between neighboring superpixels pairs along a respective one of the second paths; and
using the minimum computed second path distance as the second combined distance calculated for the unmarked superpixel.

5. The image analysis system of claim 2 , wherein the computation of the first combined distance for any of the unmarked superpixels comprises:
computing a first path distance for each of a plurality of first paths, each first path connecting the node representing the center of the unmarked superpixel and a node representing a center of one of the first marked superpixels, each first path distance being the maximum weight assigned to any one of the edges between neighboring superpixels pairs along a respective one of the first paths; and
using the minimum computed first path distance as the first combined distance calculated for the unmarked superpixel;

wherein the computation of the second combined distance for any of the unmarked superpixels comprises:
computing a second path distance for each of a plurality of second paths, each second path connecting the node representing the center of the unmarked superpixel and a node representing a center of one of the second marked superpixels, each second path distance being the maximum weight assigned to any one of the edges between neighboring superpixels pairs along a respective one of the second paths; and
using the minimum computed second path distance as the second combined distance calculated for the unmarked superpixel.

6. The image analysis system of claim 1 ,
the processor being configured for performing a pre-processing operation before the receipt of the first and second marking, the pre-processing operation comprising:
performing the identification of the plurality of superpixels;
performing, for each of the superpixels, the extraction of the feature set; and
performing the representation of the plurality of superpixels as the graph;

wherein the processor is configured for performing, after having executed the pre-processing o

Assignees

• Ventana Med Syst Inc

Inventors

• Chefd'Hotel Christophe
• Ho Stanley
• Nie Yao

Classifications

• G06T7/168

  involving transform domain methods · CPC title

• G06T2207/30024

  Cell structures in vitro; Tissue sections in vitro · CPC title

• G06T2207/10056

  Microscopic image · CPC title

• G06T2207/20156

  Automatic seed setting · CPC title

• G06T2207/20072

  Graph-based image processing · CPC title