Method, apparatus, and system using a machine learning model to segment planar regions

Patent metadata

Field	Value
Publication number	US-11710239-B2
Application number	US-202017094501-A
Country	US
Kind code	B2
Filing date	Nov 10, 2020
Priority date	Nov 10, 2020
Publication date	Jul 25, 2023
Grant date	Jul 25, 2023

Abstract

An approach is provided for using a machine learning model for identifying planar region(s) in an image. The approach involves, for example, determining the model for performing image segmentation. The model comprises at least: a trainable filter that convolves the image to generate an input volume comprising a projection of the image at different resolution scales; and feature(s) to identify image region(s) having a texture within a similarity threshold. The approach also involves processing the image using the model by generating the input volume from the image using the trainable filter and extracting the feature(s) from the input volume to determine the region(s) having the texture. The approach further involves determining the planar region(s) by clustering the image regions. The approach further involves generating a planar mask based on the planar region(s). The approach further involves providing the planar mask as an output of the image segmentation.

First claim

What is claimed is:

1. A method comprising:
determining a model for performing image segmentation to identify one or more planar regions of an image, wherein the model comprises at least:
a trainable Gaussian filter that convolves the image to generate an input volume comprising a projection of the image at a plurality of different resolution scales; wherein the same trainable Gaussian filter is used in succession to generate a plurality of images of different resolution scales; and
one or more features to identify one or more image regions having a texture within a similarity threshold;

processing the image using the model by generating the input volume from the image using the trainable filter and extracting the one or more features from the input volume to determine the one or more regions having the texture within the similarity threshold;
determining the one or more planar regions of the image by clustering the one or more image regions;
generating a planar mask based on the one or more planar regions; and
providing the planar mask as an output of the image segmentation.

2. The method of claim 1 , wherein the trainable filter is a Gaussian filter, and wherein each scale of the plurality of different resolution scales differs with respect to a blurriness level.

3. The method of claim 1 , wherein the trainable filter is a single depthwise convolutional filter repeated across one or more neural network layers of the model.

4. The method of claim 1 , wherein one or more outputs of the trainable filter are resized using a nearest-neighbor interpolation to match a spatial resolution of the image.

5. The method of claim 1 , further comprising:
performing a channelwise concatenation of one or more outputs of the trainable filter to produce the input volume from which the one or more features are extracted to determine the one or more image regions having a texture within a similarity threshold.

6. The method of claim 1 , further comprising:
constructing an image-patch to vector mapping of the image based on the one or more features,
wherein the one or more image regions are identified based on the image-patch to vector mapping.

7. The method of claim 1 , wherein the one or more planar regions of the image are determined further based on a nearest neighbor matching.

8. The method of claim 1 , wherein the model is a trained machined learning model trained through at least one source.

9. The method of claim 8 , wherein the at least one source includes a first source and a second source, wherein the first source is a final output from one or more convolutional layers of the model, and wherein the second source is the planar mask.

10. The method of claim 1 , wherein the trainable filter is an unsupervised module that provides one or more ingesting outputs to a supervised module of the model.

11. The method of claim 10 , wherein a respective custom loss function for the one or more ingesting outputs is used to train the supervised module.

12. The method of claim 1 , wherein the model is a supervised model.

13. The method of claim 1 , wherein the model is an unsupervised model.

14. An apparatus for providing a machine learning model for identifying planar region(s) in an image, comprising:
at least one processor; and
at least one memory including computer program code for one or more programs,
the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following,
determine a model for performing image segmentation to identify one or more planar regions of an image, wherein the model comprises at least:
a trainable Gaussian filter that convolves the image to generate an input volume comprising a projection of the image at a plurality of different resolution scales; wherein the same trainable Gaussian filter is used in succession to generate a plurality of images of different resolution scales; and
one or more features to identify one or more image regions having a texture within a similarity threshold;

process the image using the model by generating the input volume from the image using the trainable filter and extracting the one or more features from the input volume to determine the one or more regions having the texture within the similarity threshold;
determine the one or more planar regions of the image by clustering the one or more image regions;
generate a planar mask based on the one or more planar regions; and
provide the planar mask as an output of the image segmentation.

15. The apparatus of claim 14 , wherein the trainable filter is a Gaussian filter, and wherein each scale of the plurality of different resolution scales differs with respect to a blurriness level.

16. The apparatus of claim 14 , wherein the trainable filter is a single depthwise convolutional filter repeated across one or more neural network layers of the model.

17. The apparatus of claim 14 , wherein one or more outputs of the trainable filter are resized using a nearest-neighbor interpolation to match a spatial resolution of the image.

18. A non-transitory computer-readable storage medium for providing a machine learning model for identifying planar region(s) in an image, carrying one or more sequences of one or more instructions which, when executed by one or more processors, cause an apparatus to at least perform the following steps:
determining a model for performing image segmentation to identify one or more planar regions of an image, wherein the model comprises at least:
a trainable Gaussian filter that convolves the image to generate an input volume comprising a projection of the image at a plurality of different resolution scales; wherein the same trainable Gaussian filter is used in succession to generate a plurality of images of different resolution scales; and
one or more features to identify one or more image regions having a texture within a similarity threshold;

processing the image using the model by generating the input volume from the image using the trainable filter and extracting the one or more features from the input volume to determine the one or more regions having the texture within the similarity threshold;
determining the one or more planar regions of the image by clustering the one or more image regions;
generating a planar mask based on the one or more planar regions; and
providing the planar mask as an output of the image segmentation.

19. The computer-readable storage medium of claim 18 , wherein the trainable filter is a Gaussian filter, and wherein each scale of the plurality of different resolution scales differs with respect to a blurriness level.

20. The computer-readable storage medium of claim 18 , wherein the trainable filter is a single depthwise convolutional filter repeated across one or more neural network layers of the model.

Assignees

• Here Global Bv

Inventors

• Biswas Souham
• Boddhu Sanjay Kumar

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06T7/11 Primary

  Region-based segmentation · CPC title

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title

• G06N3/088

  Non-supervised learning, e.g. competitive learning · CPC title