Systems and methods for image matting

Patent metadata

Field	Value
Publication number	US-2018189935-A1
Application number	US-201715849379-A
Country	US
Kind code	A1
Filing date	Dec 20, 2017
Priority date	Dec 30, 2016
Publication date	Jul 5, 2018
Grant date	—

Abstract

Systems, methods, and non-transitory computer-readable media can generate an initial alpha mask for an image based on machine learning techniques. A plurality of uncertain pixels is defined in the initial alpha mask. For each uncertain pixel in the plurality of uncertain pixels, a binary value is assigned based on a nearest certain neighbor determination.

First claim

What is claimed is:

1 . A computer-implemented method comprising:
generating, by a computing system, an initial alpha mask for an image based on at least one machine learning technique; defining, by the computing system, a plurality of uncertain pixels in the initial alpha mask; and assigning, by the computing system, for at least one uncertain pixel in the plurality of uncertain pixels, a binary value based on a nearest certain neighbor determination.

2 . The computer-implemented method of claim 1 , further comprising identifying one or more edges in the image based on automated edge detection techniques.

3 . The computer-implemented method of claim 2 , wherein the nearest certain neighbor determination comprises assessing a penalty for crossing an edge of the one or more edges.

4 . The computer-implemented method of claim 3 , wherein the penalty is implemented using geodesic filtering.

5 . The computer-implemented method of claim 2 , wherein:
the assigning, for at least one uncertain pixel in the plurality of uncertain pixels, a binary value comprises assigning, for each uncertain pixel in the plurality of uncertain pixels, a binary value based on a nearest certain neighbor determination to obtain a final binary alpha mask; and the method further comprises applying an edge-aware blur to the final binary alpha mask.

6 . The computer-implemented method of claim 5 , wherein the edge-aware blur is applied to the final binary alpha mask based on guided filtering techniques.

7 . The computer-implemented method of claim 1 , further comprising:
applying a threshold filter to the initial alpha mask to obtain an initial binary alpha mask,
wherein the defining the plurality of uncertain pixels in the initial alpha mask comprises defining the plurality of uncertain pixels in the initial binary alpha mask.

8 . The computer-implemented method of claim 1 , wherein defining the plurality of uncertain pixels comprises defining a pre-defined number of uncertain pixels.

9 . The computer-implemented method of claim 8 , wherein the pre-defined number of uncertain pixels is determined based on a size of the image.

10 . The computer-implemented method of claim 1 , further comprising
generating a final alpha mask based on the assigning, for each uncertain pixel in the plurality of uncertain pixels, a binary value; and modifying the image based on the final alpha mask and a second image,
wherein
the generating the initial alpha mask comprises calculating an alpha value for each pixel in the initial alpha mask, wherein the alpha value is indicative of a likelihood that the pixel is associated with a background portion or a foreground portion, and
the alpha value is calculated based on the second image.

11 . A system comprising:
at least one processor; and a memory storing instructions that, when executed by the at least one processor, cause the system to perform a method comprising:
generating an initial alpha mask for an image based on at least one machine learning technique;
defining a plurality of uncertain pixels in the initial alpha mask; and
assigning for at least one uncertain pixel in the plurality of uncertain pixels, a binary value based on a nearest certain neighbor determination.

12 . The system of claim 11 , wherein the method further comprises:
identifying one or more edges in the image based on automated edge detection techniques.

13 . The system of claim 12 , wherein the nearest certain neighbor determination comprises assessing a penalty for crossing an edge of the one or more edges.

14 . The system of claim 13 , wherein the penalty is implemented using geodesic filtering.

15 . The system of claim 12 , wherein
the assigning, for at least one uncertain pixel in the plurality of uncertain pixels, a binary value comprises assigning, for each uncertain pixel in the plurality of uncertain pixels, a binary value based on a nearest certain neighbor determination to obtain a final binary alpha mask; and the method further comprises applying an edge-aware blur to the final binary alpha mask.

16 . A non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of a computing system, cause the computing system to perform a method comprising:
generating an initial alpha mask for an image based on at least one machine learning technique; defining a plurality of uncertain pixels in the initial alpha mask; and assigning for at least one uncertain pixel in the plurality of uncertain pixels, a binary value based on a nearest certain neighbor determination.

17 . The non-transitory computer-readable storage medium of claim 16 , wherein the method further comprises:
identifying one or more edges in the image based on automated edge detection techniques.

18 . The non-transitory computer-readable storage medium of claim 17 , wherein the nearest certain neighbor determination comprises assessing a penalty for crossing an edge of the one or more edges.

19 . The non-transitory computer-readable storage medium of claim 18 , wherein the penalty is implemented using geodesic filtering.

20 . The non-transitory computer-readable storage medium of claim 17 , wherein
the assigning, for at least one uncertain pixel in the plurality of uncertain pixels, a binary value comprises assigning, for each uncertain pixel in the plurality of uncertain pixels, a binary value based on a nearest certain neighbor determination to obtain a final binary alpha mask; and the method further comprises applying an edge-aware blur to the final binary alpha mask.

Assignees

• Facebook Inc

Inventors

• Mchugh Jason George
• Cohen Michael F
• Kopf Johannes Peter
• Dollar Piotr

Classifications

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06N20/00 Primary

  Machine learning · CPC title

• G06T7/11

  Region-based segmentation · CPC title

• G06T7/143

  involving probabilistic approaches, e.g. Markov random field [MRF] modelling · CPC title

• G06T2207/20192

  Edge enhancement; Edge preservation · CPC title