Method and apparatus for analyzing an image

Patent metadata

Field	Value
Publication number	US-11055580-B2
Application number	US-201816615568-A
Country	US
Kind code	B2
Filing date	Jun 4, 2018
Priority date	Jun 5, 2017
Publication date	Jul 6, 2021
Grant date	Jul 6, 2021

Abstract

The disclosure relates to a method and an apparatus for analyzing an image using a deep neural net pre-trained for multiple classes. The image is processed by a forward pass through an adapted neural net to generate a processing result. The adapted neural net is adapted from the pre-trained neural net to focus on exactly one selected class. The processing result is then analyzed focused on features corresponding to the selected class using an image processing algorithm. A modified image is generated by removing a manifestation of these features from the image.

First claim

The invention claimed is:

1. A method for analyzing an image having features corresponding to at least one class, the method comprising:
adapting an artificial deep neural net pre-trained for multiple classes to focus on exactly one selected class of the multiple classes by providing to pre-trained neural net training data that is labelled with the selected class, wherein the image has features corresponding to the selected class;
processing the image by a forward pass through the adapted neural net to generate a processing result that is an output of the adapted neural net after the forward pass;
analyzing the processing result to detect therein at least one object corresponding to the selected class by using an image processing algorithm to detect and segment a manifestation of the object corresponding to the selected class in the processing result; and
generating a modified image by removing the manifestation of the object corresponding to the selected class from the image,
wherein the modified image is iteratively used as input for the adapted neural net to analyze the modified image for possible remaining manifestations of the object corresponding to the selected class, and/or
wherein the pre-trained neural net is pre-trained for counting objects in images and the pre-trained neural net is adapted for counting exactly one object.

2. The method of claim 1 , wherein the processing result is taken from at least one filter of an intermediate layer of the adapted neural net and/or from an output layer of the adapted neural net.

3. The method of claim 1 , wherein the image is processed by the forward pass and a subsequent backward pass through the adapted neural net to generate the processing result.

4. The method of claim 3 , wherein the backward pass is started from an intermediary layer of the adapted neural net.

5. The method of claim 1 , wherein the artificial deep neural net is a deep convolutional neural net, a deep feedforward neural net, or a combination thereof.

6. The method of claim 1 , wherein the pre-trained neural net is adapted offline using training data.

7. The method of claim 1 , wherein, before the image is processed by the corresponding forward pass, the pre-trained neural net is adapted online using the image.

8. The method of claim 1 , wherein the artificial deep neural net is adapted using a group of images, and
wherein, after the adaptation, the group of images is sequentially processed without further adaptation steps.

9. The method of claim 1 , wherein the at least one object is removed by replacing pixel values of pixels corresponding to the at least one object with a predetermined value.

10. The method of claim 1 , wherein the pre-trained neural net is used to process the image to obtain a reference class for the image before adapting the pre-trained neural net.

11. An apparatus for analyzing an image having features corresponding to at least one class, the apparatus comprising:
an artificial deep neural net pre-trained for multiple classes; and
a separate image processing unit,
wherein the apparatus is configured to:
process the image by a forward pass through an adapted neural net to generate a processing result that is an output of the adapted neural net after the forward pass, wherein the adapted neural net is adapted from the pre-trained neural net to focus on exactly one selected class of the multiple classes by providing to pre-trained neural net training data that is labelled with the selected class, and wherein the image has features corresponding to the selected class, and
provide the processing result to the image processing unit,

wherein the image processing unit is configured to:
analyze the processing result to detect therein at least one object corresponding to the selected class by using an image processing algorithm to detect and segment a manifestation of the object corresponding to the selected class in the processing result, and
generate a modified image by removing the manifestation of the object corresponding to the selected class from the image, and

wherein the apparatus is configured to iteratively use the modified image as input for the adapted neural net to analyze the modified image for possible remaining manifestations of the object corresponding to the selected class, and/or
wherein the pre-trained neural net is pre-trained for counting objects in images and the pre-trained neural net is adapted for counting exactly one object.

Assignees

• Siemens Ag

Inventors

• Amon Peter
• Hutter Andreas
• Ghosh Sanjukta

Classifications

• G06V10/82

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/2431

  Multiple classes · CPC title

• G06V20/52 Primary

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title

• G06F18/243

  relating to the number of classes · CPC title