Image segmention via efficient semidefinate-programming based inference for binary and multi-class Markov Random Fields

Patent metadata

Field	Value
Publication number	US-11587237-B2
Application number	US-202017107437-A
Country	US
Kind code	B2
Filing date	Nov 30, 2020
Priority date	Nov 30, 2020
Publication date	Feb 21, 2023
Grant date	Feb 21, 2023

Abstract

A system for controlling a physical system via segmentation of an image includes a controller. The controller may be configured to receive an image of n pixels from a first sensor, and an annotation of the image from a second sensor, form a coupling matrix, k class vectors each of length n, and a bias coefficient based on the image and the annotation, generate n pixel vectors each of length n based on the coupling matrix, class vectors, and bias coefficient create a single segmentation vector of length n from the pixel vectors wherein each entry in the segmentation vector identifies one of the k class vectors, output the single segmentation vector; and operate the physical system based on the single segmentation vector.

First claim

What is claimed is:

1. A method of image segmentation comprising:
receiving an image of n pixels, and an annotation of the image;
forming a coupling matrix, k class vectors each of length n, and a bias coefficient based on the image and the annotation;
generating n pixel vectors each of length n based on the coupling matrix, class vectors, and bias coefficient;
creating a single segmentation vector of length n from the pixel vectors wherein each entry in the segmentation vector identifies one of the k class vectors; and
outputting the single segmentation vector.

2. The method of claim 1 , wherein the n pixel vectors are generated based utilizing received model parameters, class vectors, and a maximum number of iteration.

3. The method of claim 1 , wherein the n pixel vectors include correlation data between each pair of the n pixels.

4. The method of claim 1 , wherein each of the n pixel vectors is rounded to a predicted class, and wherein all pixel classifications are collected in a single n-dimensional segmentation vector.

5. The method of claim 4 , wherein the predicted class is one of at least 2 predicted classes of at least either a background class or one or more foreground classes.

6. The method of claim 1 , wherein the image is received from a first sensor and the annotation of the image is received from a second sensor.

7. The method of claim 6 , wherein the first sensor is an optical, light, imaging, or photon sensor.

8. The method of claim 7 , wherein the second sensor is a thermal, heat, or temperature sensor.

9. The method of claim 8 further including controlling a mechanical system based on the single segmentation vector.

10. A system for controlling a physical system via segmentation of an image comprising:
a controller configured to,
receive an image of n pixels from a first sensor, and an annotation of the image from a second sensor;
form a coupling matrix, k class vectors each of length n, and a bias coefficient based on the image and the annotation;
generate n pixel vectors each of length n based on the coupling matrix, class vectors, and bias coefficient;
create a single segmentation vector of length n from the pixel vectors wherein each entry in the segmentation vector identifies one of the k class vectors;
output the single segmentation vector; and
operate the physical system based on the single segmentation vector.

11. The system of claim 10 , wherein the first sensor is an optical, light, or photon sensor.

12. The system of claim 11 , wherein the second sensor is LIDAR, radar, sonar, thermal, heat, or temperature sensor.

13. The system of claim 12 , wherein the n pixel vectors include correlation data between each pair of the n pixels.

14. The system of claim 13 , wherein each of the n pixel vectors is rounded to a predicted class, and wherein all pixel classifications are collected in a single n-dimensional segmentation vector.

15. A system for segmenting an image for vehicle control comprising:
a first sensor configured to generate an image of n pixels;
a second sensor configured to generate an annotation of the image;
a controller configured to,
receive an image of n pixels, and an annotation of the image;
form a coupling matrix, k class vectors each of length n, and a bias coefficient based on the image and the annotation;
generate n pixel vectors each of length n based on the coupling matrix, class vectors, and bias coefficient;
create a single segmentation vector of length n from the pixel vectors wherein each entry in the segmentation vector identifies one of the k class vectors;
output the single segmentation vector; and
operate the vehicle based on the single segmentation vector.

16. The system of claim 15 , wherein first sensor is an optical, light, or photon sensor.

17. The system of claim 16 , wherein the second sensor is LIDAR, radar, sonar, thermal, heat, or a temperature sensor.

18. The system of claim 17 , wherein the n pixel vectors include correlation data between each pair of the n pixels and each of the n pixel vectors is rounded to a predicted class, and wherein all pixel classifications are collected in a single n-dimensional segmentation vector.

19. The system of claim 18 , wherein the predicted class includes one of a pedestrian, bicycle, vehicle, tree, traffic sign, traffic light, road debris, or construction barrel/cone.

20. The system of claim 18 , wherein the predicted class includes one of a lane marking, guard rail, edge of a roadway, or vehicle tracks.

Assignees

• Bosch Gmbh Robert

Inventors

• Willmott Devin T
• Pabbaraju Chirag
• Wang Po-Wei
• Kolter Jeremy

Classifications

• B60W40/10

  related to vehicle motion · CPC title

• G06T7/162 Primary

  involving graph-based methods · CPC title

• G06F18/2415

  based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate · CPC title

• B60W40/02

  related to ambient conditions · CPC title

• G06T7/143 Primary

  involving probabilistic approaches, e.g. Markov random field [MRF] modelling · CPC title