Image processing apparatus and image processing method

Patent metadata

Field	Value
Publication number	US-9704069-B2
Application number	US-201514642321-A
Country	US
Kind code	B2
Filing date	Mar 9, 2015
Priority date	Mar 10, 2014
Publication date	Jul 11, 2017
Grant date	Jul 11, 2017

Abstract

The present invention discloses an image processing evaluating apparatus and image processing method. The processing apparatus comprising: a confidence generation means for generating a classification confidence for each region in the image, the classification confidence represents the probability of an region belonging to a predefined class; and a classification means for classifying the regions in the image, which are obvious to be classified by their classification confidences, to respective classes based on the calculated confidences. The image processing apparatus further comprising: a fuzzy region extraction means for extracting one or more regions, which are not obvious to be classified by their classification confidences, as fuzzy regions; and a confidence update means for updating the classification confidence for each fuzzy region based on the classification confidences of adjacent regions thereof, wherein the classification means further classifies the fuzzy regions to respective classes based on the updated classification confidences.

First claim

What is claimed is:

1. An image processing method for classifying each of a plurality of regions in an image to one of a plurality of predefined classes using classification confidences, the method comprising:
a confidence generation step for generating, for each of a plurality of regions of the image, a set of classification confidences, each classification confidence corresponding to one of a plurality of predefined classes and representing the probability of the respective region belonging to the predefined class; and
a classification step for classifying the regions in the image wherein for each of the regions for which an obvious match with one of the predefined classes exists based on the classification confidences associated with the region, classifying the region to the respective predefined class based on the classification confidences associated with the region,
the method further comprising:
a fuzzy region extraction step for extracting and designating as fuzzy regions one or more regions of the plurality of regions, which do not have an obvious match with one of the plurality of predefined classes based on their set of associated classification confidences; and
a confidence update step for updating the classification confidence for each such fuzzy region based on the sets of classification confidences associated with adjacent regions thereof,
wherein the classification step further classifies the fuzzy regions to respective predefined classes based on the updated classification confidences.

2. The image processing method according to claim 1 , wherein the fuzzy region extraction step and the confidence update step are repeated one or more times.

3. The image processing method according to claim 1 , wherein the fuzzy region extraction step comprises:
a confidence normalization step for normalizing the classification confidences for each region in the image;
a confidence difference calculation step for calculating the difference between the maximum classification confidence and the second maximum classification confidence for each region; and
a fuzzy region judgment step for judging a region, for which the calculated confidence difference is lower than a predefined threshold, to be a fuzzy region.

4. The image processing method according to claim 1 , wherein the fuzzy region extraction step comprises:
a confidence map generation step for creating a confidence map for each predefined class based on the classification confidence of each region;
a region segmentation step for segmenting each confidence map into foreground and background regions of each predefined class based on the confidence maps and predefined thresholds; and
a fuzzy region judgment step for judging a region, which is not segmented into the foreground region of one and only one predefined class, to be a fuzzy region.

5. The image processing method according to claim 3 , wherein
the fuzzy region extraction step further comprises a fuzzy region merging step for merging the adjacent fuzzy regions.

6. The image processing method according to claim 4 , wherein each confidence map is segmented into foreground and background regions of each predefined class based on Graph-cut method in the region segmentation step.

7. The image processing method according to claim 1 , wherein the confidence update step comprises:
an adjacent map establishment step for obtaining information about the neighborhood of each fuzzy region;
a weighted confidence calculating step for accumulating the classification confidences associated with each fuzzy region and the adjacent regions thereof in weighting; and
a confidence normalization step for normalizing the accumulated classification confidence of each fuzzy region as the updated classification confidence.

8. The image processing method according to claim 5 , wherein the confidence update step comprises:
an adjacent map establishment step for obtaining information about the neighborhood of each merged fuzzy region obtained in the fuzzy region extraction step;
a weighted confidence calculating step for accumulating the sets of classification confidences associated with the fuzzy regions in each merged fuzzy region and the adjacent regions of the merged fuzzy region in weighting; and
a confidence normalization step for normalizing the accumulated classification confidence of each fuzzy region as the updated classification confidence.

9. The image processing method according to claim 1 , wherein the confidence generation step comprises:
a region segmentation step for segmenting the image into a plurality of non-overlapped regions;
a feature extraction step for extracting the features for each region; and
a confidence calculation step for calculating the classification confidence for each region based on a trained model for each predetermined class.

10. The image processing method according to claim 1 , wherein the obtained classifications of the regions of the image are used to perform an image composition or an image search.

11. An image processing apparatus for classifying each of a plurality of regions in an image to one of a plurality of predefined classes using classification confidences, the apparatus comprising:
a confidence generation means for generating, for each of a plurality of regions of the image, a set of classification confidences, each classification confidence corresponding to one of a plurality of predefined classes and representing the probability of the respective region belonging to the predefined class; and
a classification means for classifying the regions in the image, wherein for each of the regions of the image for which an obvious match with one of the predefined classes exists based on the classification confidences associated with the region, classifying the region to the respective predefined class based on the classification confidences associated with the region,
the image processing apparatus further comprising:
a fuzzy region extraction means for extracting and designating as fuzzy regions one or more regions of the plurality of regions, which do not have an obvious match with one of the plurality of predefined classes based on their set of associated classification confidences; and
a confidence update means for updating the classification confidence for each such fuzzy region based on the sets of classification confidences associated with adjacent regions thereof,
wherein the classification means further classifies the fuzzy regions to respective predefined classes based on the updated classification confidences.

12. The image processing apparatus according to claim 11 , wherein the fuzzy region extraction means comprises:
a confidence normalization unit for normalizing the classification confidences for each region in the image;
a confidence difference calculation unit for calculating the difference between the maximum classification confidence and the second maximum classification confidence for each region; and
a fuzzy region judgment unit for judging a region, for which the calculated confidence difference is lower than a predefined threshold, as a fuzzy region.

13. The image processing apparatus according to claim 11 , wherein the fuzzy region extraction means comprises:
a confidence map generation unit for creating a confidence map for each predefined class based on the classification confidence of each region;
a region segmentation unit for segmenting each confidence map into foreground and background regions of each predefined class based on the confidence maps and predefined thresholds; and
a fuzzy region judgment unit for judging region, which is not segmented into the for

Assignees

• Canon Kk

Inventors

• Lv you
• Jiang Yong
• Wu Bo

Classifications

• G06V20/20 Primary

  in augmented reality scenes · CPC title

• G06V10/768

  using context analysis, e.g. recognition aided by known co-occurring patterns · CPC title

• G06K9/72 Primary

  Physics · mapped topic

• G06K9/00671

  Physics · mapped topic