Method and system for impurity detection using multi-modal imaging

Patent metadata

Field	Value
Publication number	US-11270429-B2
Application number	US-202016900106-A
Country	US
Kind code	B2
Filing date	Jun 12, 2020
Priority date	Jun 13, 2019
Publication date	Mar 8, 2022
Grant date	Mar 8, 2022

Abstract

The disclosure herein generally relates to image processing, and, more particularly, to a method and system for impurity detection using multi-modal image processing. This system uses a combination of polarization data, and at least one of a depth data and an RGB image data to perform the impurity material detection. The system uses a graph fusion based approach while processing the captured images to detect presence of the impurity material, and accordingly alert the user.

First claim

What is claimed is:

1. A processor implemented method of image processing for impurity identification, comprising:
obtaining, via one or more hardware processors, an RGB-P (Red-Green-Blue Polarization) image and at least one of an RGB (Red-Green-Blue) image, and RGB-D (Red-Green-Blue Depth) image, of a scene, as inputs;
pre-processing the obtained RGB-P image and the at least one RGB and RGB-D image to remove motion blur, via the one or more hardware processors;
generating a Degree of Polarization (DoP) map from at least one plane of the RGB-P image, via the one or more hardware processors;
constructing a first graph and at least one of a second graph and a third graph, via the one or more hardware processors, wherein the first graph is constructed for the DoP map, the second graph is constructed for the RGB image, and the third graph is constructed for the RGB-D image;
constructing a joint graph spectrum by merging the first graph, and the at least one of the second graph and the third graph, via the one or more hardware processors;
performing spectral clustering using the joint graph spectrum, comprising:
generating a plurality of segments from the joint graph spectrum; and
mapping values of each of the plurality of segments from the DoP map; and

determining, based on size of each of the plurality of segments and mapped value of each of the plurality of segments, presence or absence of one or more impurity materials in the scene, via the one or more hardware processors.

2. The processor implemented method as claimed in claim 1 , wherein, the joint graph spectrum is constructed such that overlap of the joint graph spectrum and a graph spectrum of the first graph exceeds a threshold of overlap, and the joint graph spectrum is smooth with respect to the second graph and the third graph.

3. The processor implemented method as claimed in claim 1 , wherein determining the presence or absence of the one or more impurity materials based on the size of the segments and the values of the segments comprises:
comparing the size and values of each of the segments with a threshold of size of segments and a threshold of polarization values of segments respectively; and
determining the presence of the one or more impurity materials if at least one of the size of the segments or value of segments exceed the threshold of size of segments and a threshold of polarization values of segments.

4. A system of image processing for impurity identification, comprising:
one or more hardware processors;
one or more communication interfaces; and
a memory, wherein the memory comprises a plurality of instructions, which when executed, cause the one or more hardware processors to:
obtain an RGB-P (Red-Green-Blue Polarization) image and at least one of an RGB (Red-Green-Blue) image, and RGB-D (Red-Green-Blue Depth) image, of a scene, as inputs;
pre-process the obtained RGB-P image and the at least one RGB and RGB-D image to remove motion blur;
generate a Degree of Polarization (DoP) map from at least one plane of the RGB -P image;
construct a first graph and at least one of a second graph and a third graph, wherein the first graph is constructed for the DoP map, the second graph is constructed for the RGB image, and the third graph is constructed for the RGB-D image;
construct a joint graph spectrum by merging the first graph, and the at least one of the second graph and the third graph;
perform spectral clustering using the joint graph spectrum, by:
generating a plurality of segments from the joint graph spectrum; and
mapping values of each of the plurality of segments from the DoP map; and

determine based on size of each of the segments and mapped value of each of the segments, presence or absence of one or more impurity materials in the scene.

5. The system as claimed in claim 4 , wherein the system constructs the joint graph spectrum such that overlap of the joint graph spectrum and a graph spectrum of the first graph exceeds a threshold of overlap, and the joint graph spectrum is smooth with respect to the second graph and the third graph.

6. The system as claimed in claim 4 , wherein the system determines the presence or absence of the one or more impurity materials based on the size of the segments and the values of the segments by:
comparing the size and values of each of the segments with a threshold of size of segments and a threshold of polarization values of segments respectively; and
determining the presence of the one or more impurity materials if at least one of the size of the segments or value of segments exceed the threshold of size of segments and a threshold of polarization values of segments.

7. A non-transitory computer readable medium for image processing for impurity identification, wherein the non-transitory computer readable medium performs the image processing by:
obtaining, via one or more hardware processors, an RGB-P (Red-Green-Blue Polarization) image and at least one of an RGB (Red-Green-Blue) image, and RGB-D (Red-Green-Blue Depth) image, of a scene, as inputs;
pre-processing the obtained RGB-P image and the at least one RGB and RGB-D image to remove motion blur, via the one or more hardware processors;
generating a Degree of Polarization (DoP) map from at least one plane of the RGB -P image, via the one or more hardware processors;
constructing a first graph and at least one of a second graph and a third graph, via the one or more hardware processors, wherein the first graph is constructed for the DoP map, the second graph is constructed for the RGB image, and the third graph is constructed for the RGB-D image;
constructing a joint graph spectrum by merging the first graph, and the at least one of the second graph and the third graph, via the one or more hardware processors;
performing spectral clustering using the joint graph spectrum, comprising:
generating a plurality of segments from the joint graph spectrum; and
mapping values of each of the plurality of segments from the DoP map; and

determining, based on size of each of the plurality of segments and mapped value of each of the plurality of segments, presence or absence of one or more impurity materials in the scene, via the one or more hardware processors.

8. The non-transitory computer readable medium as claimed in claim 7 , wherein the non-transitory computer readable medium constructs the joint graph spectrum such that overlap of the joint graph spectrum and a graph spectrum of the first graph exceeds a threshold of overlap, and the joint graph spectrum is smooth with respect to the second graph and the third graph.

9. The non-transitory computer readable medium as claimed in claim 7 , wherein the non-transitory computer readable medium determines the presence or absence of the one or more impurity materials based on the size of the segments and the values of the segments, comprises:
comparing the size and values of each of the segments with a threshold of size of segments and a threshold of polarization values of segments respectively; and
determining the presence of the one or more impurity materials if at least one of the size of the segments or value of segments exceed the threshold of size of segments and a threshold of polarization values of segments.

Assignees

• Tata Consultancy Services Ltd

Inventors

• Kumar Achanna Anil
• Khawad Rishab
• Panse Riddhi
• Gigie Andrew
• Chakravarty Tapas
• Kumar Kriti
• Sahu Saurabh
• Chandra Mariswamy Girish

Classifications

• G06T7/11

  Region-based segmentation · CPC title

• G06T2207/10024

  Color image · CPC title

• G06T7/0008 Primary

  checking presence/absence · CPC title

• G01N21/8851

  Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges (G01N21/8806 and G01N21/93 - G01N21/95692 take precedence; optical measurement of dimensions G01B11/00; optical scanning G02B26/10; image transformation G06T3/00; computerised image enhancement G06T5/00; image processing per se for flaw detection G06T7/0002) · CPC title

• G06T7/187

  involving region growing; involving region merging; involving connected component labelling · CPC title