Image processing via isotonic convolutional neural networks

Patent metadata

Field	Value
Publication number	US-11922291-B2
Application number	US-202117487631-A
Country	US
Kind code	B2
Filing date	Sep 28, 2021
Priority date	Sep 28, 2021
Publication date	Mar 5, 2024
Grant date	Mar 5, 2024

Abstract

A convolutional neural network system includes a sensor and a controller, wherein the controller is configured to receive an image from the sensor, divide the image into patches, each patch of size p, extract, via a first convolutional layer, a feature map having a number of channels based on a feature detector of size p, wherein the feature detector has a stride equal to size p, refine the feature map by alternatingly applying depth-wise convolutional layers and point-wise convolutional layers to obtain a refined feature map, wherein the number of channels in the feature map and the size of the feature map remains constant throughout all operations in the refinement; and output the refined feature map.

First claim

What is claimed is:

1. A computer-implemented method for processing an image utilizing a convolutional neural network, the computer-implemented method comprising:
receiving an image;
dividing the image into patches, each patch of size p;
extracting, via a first convolutional layer, a feature map having a number of channels based on a feature detector of size p, wherein the feature detector has a stride equal to size p;
refining the feature map by alternatingly applying depth-wise convolutional layers and point-wise convolutional layers to obtain a refined feature map, wherein the number of channels in the feature map, and the size of the feature map remains constant throughout all operations in the refinement; and
outputting the refined feature map.

2. The method of claim 1 , wherein the method includes receiving the image from a sensor.

3. The method of claim 2 , wherein the sensor is one of video, RADAR, LIDAR, or ultrasound, and is in communication with a controller is configured to control an autonomous vehicle based on the refined feature map.

4. The method of claim 2 , wherein the sensor is one of video, sound, IR, or LIDAR, and is in communication with a controller configured to control an access door based on the refined feature map.

5. The method of claim 2 , wherein the sensor is one of video, sound, ultrasound, IR, or LIDAR, and is in communication with a controller configured to control a mechanical system based on the refined feature map.

6. The method of claim 1 , wherein the first convolutional layer includes an activation function that is a Gaussian Error Linear Unit (GELU).

7. The method of claim 1 further comprising, averaging the feature map over spatial locations for each channel to obtain a mean for all channels;
transforming the mean for all channels to obtain a probability that the input image corresponds to a specific class; and
outputting the probability that the image belongs to the specific class.

8. A computer-implemented method for processing an image utilizing a convolutional neural network, the computer-implemented method comprising:
receiving an image of size L×W;
dividing the image into patches, wherein a combined size of each patch equals L×W;
extracting, via a first convolutional layer, a feature map having a number of channels based on a feature detector of size equal to the patch size, wherein the feature detector has a stride equal to the patch size;
refining the feature map by alternatingly applying depth-wise convolutional layers and point-wise convolutional layers to obtain a refined feature map, wherein the number of channels in the feature map and the size of the feature map remains constant throughout all operations in the refinement; and
outputting the refined feature map.

9. The method of claim 8 , wherein each patch size is p×p.

10. The method of claim 8 , wherein the method includes receiving the image from a sensor.

11. The method of claim 10 , wherein the sensor is one of video, RADAR, LIDAR, or ultrasound, and in communication with a controller configured to control an autonomous vehicle based on the refined feature map.

12. The method of claim 10 , wherein the sensor is one of video, sound, IR, or LIDAR, and in communication with a controller configured to control an access door based on the refined feature map.

13. The method of claim 10 , wherein the first sensor is one of video, sound, ultrasound, IR, or LIDAR, and in communication with a controller configured to control a mechanical system based on the refined feature map.

14. The method of claim 8 further comprising, averaging the feature map over spatial locations for each channel to obtain a mean for all channels;
transforming the mean for all channels to obtain a probability that the input image corresponds to a specific class; and
outputting the probability that the image belongs to the specific class.

15. A convolutional neural network system comprising:
a sensor; and
a controller, wherein the controller is configured to
receive an image from the sensor,
divide the image into patches, each patch of size p,
extract, via a first convolutional layer, a feature map having a number of channels based on a feature detector of size p, wherein the feature detector has a stride equal to size p,
refine the feature map by alternatingly applying depth-wise convolutional layers and point-wise convolutional layers to obtain a refined feature map, wherein the number of channels in the feature map and the size of the feature map remains constant throughout all operations in the refinement; and
output the refined feature map.

16. The convolutional neural network system of claim 15 , wherein the sensor is one of video, RADAR, LIDAR, or ultrasound, and the controller is further configured to control an autonomous vehicle based on the refined feature map.

17. The convolutional neural network system of claim 15 , wherein the sensor is one of video, sound, IR, or LIDAR, and the controller is further configured to control an access door based on the refined feature map.

18. The convolutional neural network system of claim 15 , wherein the sensor is one of video, sound, ultrasound, IR, or LIDAR, and the controller is further configured to control a mechanical system based on the refined feature map.

19. The convolutional neural network system of claim 15 wherein the controller is further configured to, average the feature map over spatial locations for each channel to obtain a mean for all channels;
transform the mean for all channels to obtain a probability that the input image corresponds to a specific class; and
output the probability that the image belongs to the specific class.

Assignees

• Bosch Gmbh Robert
• Univ Carnegie Mellon

Inventors

• Trockman Asher
• Kolter Jeremy
• Willmott Devin T
• Cabrita Condessa Filipe J

Classifications

• G06N3/047 Primary

  Probabilistic or stochastic networks · CPC title

• G06F18/2431 Primary

  Multiple classes · CPC title

• G06T7/11

  Region-based segmentation · CPC title

• G06T2207/10044

  Radar image · CPC title

• G06T2207/10132

  Ultrasound image · CPC title