Image analysis including targeted preprocessing

Patent metadata

Field	Value
Publication number	US-11087470-B2
Application number	US-201816628320-A
Country	US
Kind code	B2
Filing date	Jun 11, 2018
Priority date	Jul 4, 2017
Publication date	Aug 10, 2021
Grant date	Aug 10, 2021

Abstract

A system includes a K1 preprocessing module designed to generate at least one intermediate image from an input image using a parameterized internal processing chain and an analysis module to detect a feature or object in the intermediate image. A method to train the system includes feeding a plurality of learning input images to the system, comparing a result provided by the analysis module for each of the learning input images to a learning value, and feeding back a deviation obtained by the comparison to an input preprocessing module and/or adapting parameters of the internal processing chain to reduce the deviation.

First claim

What is claimed is:

1. A system comprising:
at least one artificial intelligence preprocessing module that includes a parameterized internal processing chain including at least one artificial neural network composed of a plurality of layers of artificial neurons that are configured to generate at least one intermediate image from an input image; and
a plurality of analysis modules;
wherein:
the plurality of layer include an input layer, an intermediate image layer, and at least two processing layers between the input and intermediate image layers;
the input layer is configured to record the input image;
the intermediate image layer is configured to store the intermediate image;
parameters of the processing chain includes weights of connections between the plurality of layers of the neural network; and
at least one of the plurality of analysis modules is configured to detect at least one object in the at least one intermediate image while at least one other of the plurality of analysis modules is configured to detect at least one edge in the at least one intermediate image.

2. The system of claim 1 , further comprising at least one assessment module, wherein the at least one assessment module is configured to qualitatively assess, and feed back as an input to the preprocessing module, respective results provided by the analysis modules.

3. The system of claim 2 , wherein the assessment module is configured to combine the assessments of the respective results provided by the plurality of analysis modules into an overall assessment and feed the overall assessment back as input to the preprocessing module.

4. The system of claim 3 , wherein, in the preprocessing module, the input image passes through one and the same internal processing chain during generation of a plurality of instances of the intermediate image respectively for the plurality of analysis modules.

5. The system of claim 1 , wherein at least one of the layers is configured to convolve image data using a convolution kernel, and/or compress the image data by downsampling.

6. The system of claim 1 , wherein the plurality of layers include, on a side of the processing chain opposite of the input layer, at least one analysis layer in which at least a part of a functionality of the analysis modules is implemented.

7. The system of claim 1 , further comprising a camera by which the input image is obtained.

8. The system of claim 7 , wherein the camera is adapted to be cooled by a windshield of a vehicle, and the camera has an electric power consumption of maximally five watts and/or the preprocessing module is adapted for ascertaining at least one controlled variable that is fed to the camera for at least one operating parameter thereof.

9. The system of claim 1 , wherein the system is coupled to, and is configured to convert at least one detection of the edge or object into a control signal that acts on, a steering system, a braking system, and/or a drive system of an at least partially automated vehicle.

10. The system of claim 1 , wherein the artificial intelligence preprocessing module is a K1 preprocessing module.

11. A method for training a system that includes (a) at least one artificial intelligence preprocessing module that includes a parameterized internal processing chain including at least one artificial neural network composed of a plurality of layers of artificial neurons that are configured to generate at least one intermediate image from an input image, and (b) a plurality of analysis modules, wherein (1) the plurality of layer include an input layer, an intermediate image layer, and at least two processing layers between the input and intermediate image layers, (2) the input layer is configured to record the input image, (3) the intermediate image layer is configured to store the intermediate image, (4) parameters of the processing chain includes weights of connections between the plurality of layers of the neural network, and at least one of the plurality of analysis modules is configured to detect at least one object in the at least one intermediate image while at least one other of the plurality of analysis modules is configured to detect at least one edge in the at least one intermediate image, the method comprising:
feeding a plurality of learning input images to the system;
comparing results provided by the analysis modules for each of the learning input images to respective learning values; and
feeding back a deviation provided by the comparing to an input of the preprocessing module and/or adapting the parameters of the processing chain to reduce the deviation.

12. The method of claim 11 , wherein the method includes the adapting, and a gradient descent method is used for the adapting.

13. A non-transitory computer-readable medium on which are stored instructions that are exectutable by a computer and that, when executed by the computer, cause the computer to form a system that includes:
at least one artificial intelligence preprocessing module that includes a parameterized internal processing chain including at least one artificial neural network composed of a plurality of layers of artificial neurons that are configured to generate at least one intermediate image from an input image; and
a plurality of analysis modules;
wherein:
the plurality of layer include an input layer, an intermediate image layer, and at least two processing layers between the input and intermediate image layers;
the input layer is configured to record the input image;
the intermediate image layer is configured to store the intermediate image;
parameters of the processing chain includes weights of connections between the plurality of layers of the neural network; and
at least one of the plurality of analysis modules is configured to detect at least one object in the at least one intermediate image while at least one other of the plurality of analysis modules is configured to detect at least one edge in the at least one intermediate image.

14. A non-transitory computer-readable medium on which are stored instructions that are executable by a processor and that, when executed by the processor, cause the processor to perform a method that trains a system that includes (a) at least one artificial intelligence preprocessing module that includes a parameterized internal processing chain including at least one artificial neural network composed of a plurality of layers of artificial neurons that are configured to generate at least one intermediate image from an input image, and (b) a plurality of analysis modules, wherein (1) the plurality of layer include an input layer, an intermediate image layer, and at least two processing layers between the input and intermediate image layers, (2) the input layer is configured to record the input image, (3) the intermediate image layer is configured to store the intermediate image, (4) parameters of the processing chain includes weights of connections between the plurality of layers of the neural network, and at least one of the plurality of analysis modules is configured to detect at least one object in the at least one intermediate image while at least one other of the plurality of analysis modules is configured to detect at least one edge in the at least one intermediate image, the method comprising:
feeding a plurality of learning input images to the system;
comparing results provided by the analysis modules for each of the learning input images to respective learning values; and
feeding back a deviation provided by the comparing to an input of the preprocessing module and/or adapting the parameters of the processing chain to reduce

Assignees

• Bosch Gmbh Robert

Inventors

• Mehnert Jens Eric Markus
• Bischoff Stefan
• Lenor Stephan

Classifications

• G06V10/7753

  Incorporation of unlabelled data, e.g. multiple instance learning [MIL] · CPC title

• G06V10/30

  Noise filtering · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• G06T7/13 Primary

  Edge detection · CPC title