Using activation maps to detect best areas of an image for prediction of noise levels

Patent metadata

Field	Value
Publication number	US-12567233-B2
Application number	US-202118035719-A
Country	US
Kind code	B2
Filing date	Nov 8, 2021
Priority date	Nov 9, 2020
Publication date	Mar 3, 2026
Grant date	Mar 3, 2026

Abstract

An image processing apparatus is provided which obtains and provides image data at a first scale as input to a first classifier trained based on images in the first scale to classify the image data in a first class or a second class, outputs, from the first classifier, activation map data and image array data, obtains at least target region of the image data at the first scale based on output of a second classifier that uses the activation map data and image array data from the first classifier, maps the at least one target region to image data at a second scale, extracts target region image data from each of the at least one target region of the image data at the second scale and classifies, as a first type of image or a second type of image.

First claim

I claim:

1 . An image processing method comprising:
obtaining image data stored in memory of a processing device; providing the image data at a first scale as input to a first classifier, the first classifier being trained based on images in the first scale to classify the image data in a first class or a second class; outputting, from the first classifier, activation map data and image array data; obtaining at least one target region of the image data at the first scale based on output of a second classifier that uses the activation map data and image array data from the first classifier; mapping the at least one target region to image data at a second scale; extracting target region image data from each of the at least one target region of the image data at the second scale; classifying, as a first type of image or a second type of image, the obtained image data based on the extracted target region image data using a third classifier trained using cropped image data at the second scale to estimate noise.

2 . The image processing method according to claim 1 wherein the classifying, as a first type or second type of image is performed by
calculating an average by predicted class of each of the at least one target regions over an entire area of the obtained image data; and
labeling the obtained image data as the first type or second type based on the calculated average.

3 . The image processing method of claim 1 , wherein the obtaining at least one target region further comprises identifying coordinate locations within the obtained image data based on the image array data and the activation map such that each of the target region represents a maximum value.

4 . The image processing method of claim 3 , wherein extracting target region image data further comprises:
using each of the identified coordinate locations as a center point; and generating a bounding box having a predetermine size around each center point; and extracting, as the target region image data, the image data within each generated bounding box.

5 . The image processing method of claim 1 , further comprising
outputting, on a display, the obtained image data including the label identifying the image as the first type of image or the second type of image.

6 . The image processing method of claim 5 , wherein the first type of image is an image classified as noisy and the second type of image is an image classified as non-noisy.

7 . An image processing apparatus comprising:
one or more memories storing instructions; and one or more processors that, upon execution of the instructions, is configured to perform operations including:
obtaining image data stored in memory of a processing device;
providing the image data at a first scale as input to a first classifier, the first classifier being trained based on images in the first scale to classify the image data in a first class or a second class;
outputting, from the first classifier, activation map data and image array data;
obtaining at least one target region of the image data at the first scale based on output of a second classifier that uses the activation map data and image array data from the first classifier;
mapping the at least one target region to image data at a second scale;
extracting target region image data from each of the at least one target region of the image data at the second scale;
classifying, as a first type of image or a second type of image, the obtained image data based on the extracted target region image data using a third classifier trained using cropped image data at the second scale to estimate noise.

8 . The image processing apparatus according to claim 7 , wherein execution of the instructions further configures the one or more processors to perform operations comprising
classifying, as a first type or second type of image is performed by calculating an average by predicted class of each of the at least one target regions over an entire area of the obtained image data; and labeling the obtained image data as the first type or second type based on the calculated average.

9 . The image processing apparatus of claim 7 , wherein execution of the instructions further configures the one or more processors to perform operations comprising
obtaining at least one target region by identifying coordinate locations within the obtained image data based on the image array data and the activation map such that each of the target region represents a maximum value.

10 . The image processing apparatus of claim 9 , wherein execution of the instructions further configures the one or more processors to perform operations comprising extracting target region image data by:
using each of the identified coordinate locations as a center point; and generating a bounding box having a predetermine size around each center point; and extracting, as the target region image data, the image data within each generated bounding box.

11 . The image processing apparatus of claim 7 , wherein execution of the instructions further configures the one or more processors to perform operations comprising
outputting, on a display, the obtained image data including the label identifying the image as the first type of image or the second type of image.

12 . The image processing apparatus of claim 11 , wherein the first type of image is an image classified as noisy and the second type of image is an image classified as non-noisy.

Assignees

• Canon Usa Inc

Inventors

• Guyduy Yevgeniy Gennadiy

Classifications

• G06T2207/20132

  Image cropping · CPC title

• G06T7/0002

  Inspection of images, e.g. flaw detection · CPC title

• G06V20/70

  Labelling scene content, e.g. deriving syntactic or semantic representations · CPC title

• G06V10/25

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title