Object detection in image data using color segmentation

Patent metadata

Field	Value
Publication number	US-10235594-B2
Application number	US-201615364045-A
Country	US
Kind code	B2
Filing date	Nov 29, 2016
Priority date	Nov 29, 2016
Publication date	Mar 19, 2019
Grant date	Mar 19, 2019

Abstract

Image color data for a field of view is received. Thereafter, color segmentation can be performed on the image color data to define at least one bounding polygon that minimizes an amount of free space within each bounding polygon. The at least one bounding polygon is then used to crop the image color data to result in cropped image color data. Image processing can then be applied to the cropped image color data to identify at least one object therein. Related apparatus, systems, techniques and articles are also described.

First claim

What is claimed is:

1. A method for enhanced object localization and object characterization within multi-dimensional image data, the method being implemented by one or more data processors forming part of at least one computing device and comprising:
receiving multi-dimensional, digital image color data for a field of view;
performing color segmentation on the image color data to define at least one bounding polygon that minimizes an amount of free space within each bounding polygon, the color segmentation comprising a sequence including converting color images within the image color data into binary images, selectively inverting the binary images, and converting the binary images back into color images;
cropping using the at least one bounding polygon, the image color data to result in cropped image color data; and
applying image processing to the cropped image color data to identify at least one object therein;
wherein:
selectively inverting the binary images comprises generating a discarded connected regions image and inverting the discarded connected regions image;
the at least one bounding polygon is formed around a biggest connected region within the inverted discarded connected regions image; and
the performing color segmentation comprises:
cropping the image color data using the at least one bounding polygon to result in a cropped RGB image;
performing color thresholding on the cropped RGB image to consolidate colors within pre-defined color ranges to result in a color threshold image;
defining edges for the color threshold image to result in an edge image;
filling any holes within an area encapsulated by the defined edges to result in a hole-filled edge image;
filling at least a portion of the area encapsulated by the defined edges in the hole-filled edge image in black to generate a filled with black image;
converting the filled with black image to a binary image comprising solely black and white pixels;
inverting the binary image to result in a complementary binary image;
filtering out regions of pixels in the binary image having an area below a pre-defined threshold to result in a small regions filter image;
inverting the small regions filter image and discarding connected regions of pixels identified in the inverted small regions filter image having an area below a pre-defined threshold to result in the discarded connected regions image; and
inverting the discarded connected regions image and forming the at least one bounding polygon around the biggest connected region within the inverted discarded connected regions image.

2. The method of claim 1 , wherein a shape of the at least one bounding polygon is rectangular.

3. The method of claim 1 , wherein a shape of the at least one bounding polygon has three or more sides.

4. The method of claim 1 , wherein the image color data is RGB data.

5. A system for enhanced object localization and object characterization within multi-dimensional image data comprising:
at least one data processor;
memory storing instructions which, when executed by the at least one data processor, result in operations comprising:
receiving multi-dimensional, digital image color data for a field of view;
performing color segmentation on the image color data to define at least one bounding polygon that minimizes an amount of free space within each bounding polygon, the color segmentation comprising a sequence including converting color images within the image color data into binary images, selectively inverting the binary images, and converting the binary images back into color images;
cropping, using the at least one bounding polygon, the image color data to result in cropped image color data; and
applying image processing to the cropped image color data to identify at least one object therein;

wherein:
selectively inverting the binary images comprises generating a discarded connected regions image and inverting the discarded connected regions image;
the at least one bounding polygon is formed around a biggest connected region within the inverted discarded connected regions image; and
the performing color segmentation comprises:
cropping the image color data using the at least one bounding polygon to result in a cropped RGB image;
performing color thresholding on the cropped RGB image to consolidate colors within pre-defined color ranges to result in a color threshold image;
defining edges for the color threshold image to result in an edge image;
filling any holes within an area encapsulated by the defined edges to result in a hole-filled edge image;
filling at least a portion of the area encapsulated by the defined edges in the hole-filled edge image in black to generate a filled with black image;
converting the filled with black image to a binary image comprising solely black and white pixels;
inverting the binary image to result in a complementary binary image;
filtering out regions of pixels in the binary image having an area below a pre-defined threshold to result in a small regions filter image;
inverting the small regions filter image and discarding connected regions of pixels identified in the inverted small regions filter image having an area below a pre-defined threshold to result in the discarded connected regions image; and
inverting the discarded connected regions image and forming the at least one bounding polygon around the biggest connected region within the inverted discarded connected regions image.

6. The system of claim 5 , wherein a shape of the at least one bounding polygon has three or more sides.

7. A non-transitory computer program product for enhanced object localization and object characterization within multi-dimensional image data, the computer program product storing instructions which, when executed by at least one data processor forming part of at least one computing device, result in operations comprising:
receiving multi-dimensional, digital image color data for a field of view;
performing color segmentation on the image color data to define at least one bounding polygon that minimizes an amount of free space within each bounding polygon, the color segmentation comprising a sequence including converting color images within the image color data into binary images, selectively inverting the binary images, and converting the binary images back into color images;
cropping, using the at least one bounding polygon, the image color data to result in cropped image color data; and
applying image processing to the cropped image color data to identify at least one object therein;
wherein the performing color segmentation comprises:
cropping the image color data using the at least one bounding polygon to result in a cropped RGB image;
performing color thresholding on the cropped RGB image to consolidate colors within pre-defined color ranges to result in a color threshold image;
defining edges for the color threshold image to result in an edge image;
filling any holes within an area encapsulated by the defined edges to result in a hole-filled edge image;
filling at least a portion of the area encapsulated by the defined edges in the hole-filled edge image in black to generate a filled with black image;
converting the filled with black image to a binary image comprising solely black and white pixels;
inverting the binary image to result in a complementary binary image;
filtering out regions of pixels in the binary image having an area below a pre-defined threshold to result in a small regions filter image;
inverting the small regions filter image and discarding connected regions of pixels identified in the inverted small regions filter image having an area below a pre-defined threshold to result in a discarded connected regions image; and
inverting t

Assignees

• Sap Se

Inventors

• Farooqi Waqas Ahmad
• Lipps Jonas
• Schmidt Eckehard
• Fricke Thomas
• Verzano Nemrude

Classifications

• G06V10/809

  of classification results, e.g. where the classifiers operate on the same input data · CPC title

• G06V10/56

  relating to colour · CPC title

• G06T7/11 Primary

  Region-based segmentation · CPC title

• G06F18/254

  of classification results, e.g. of results related to same input data · CPC title

• G06V10/255

  Detecting or recognising potential candidate objects based on visual cues, e.g. shapes · CPC title