Image synthesis utilizing an active mask

Patent metadata

Field	Value
Publication number	US-9710898-B2
Application number	US-201514945308-A
Country	US
Kind code	B2
Filing date	Nov 18, 2015
Priority date	Nov 18, 2015
Publication date	Jul 18, 2017
Grant date	Jul 18, 2017

Abstract

Embodiments of the present invention provide systems, methods, and computer storage media directed at image synthesis utilizing an active mask. In one embodiment, input is received that identifies a target region within an image that is to be synthesized. A patch synthesis technique can then be performed to synthesize the target region based on portions of a source region that are identified by the patch synthesis technique. In embodiments, the patch synthesis technique includes, for at least one iteration, generating an active mask that indicates one or more portions of the target region as inactive. This active mask can be utilized by at least one process of the patch synthesis technique to ignore the one or more portions indicated as inactive by the active mask for the at least one iteration of the patch synthesis technique. Other embodiments may be described and/or claimed.

First claim

What is claimed is:

1. One or more non-transitory computer-readable storage media having instructions stored thereon, which, when executed by one or more processors of a computing device, provide the computing device with a graphics editor to:
receive input identifying a target region within an image, wherein the target region is to be synthesized;
perform a patch synthesis technique to synthesize the target region based on portions of a source region within the image, wherein the portions of the source region are identified by the patch synthesis technique, at least one iteration of the patch synthesis technique including:
identification of groupings of pixels that have converged in at least one other iteration of the patch synthesis technique; and
generation of an active mask that indicates one or more portions of the target region as inactive, the one or more portions of the target region being composed of the groupings of pixels that have converged in the at least one other iteration of the patch synthesis technique, wherein at least one process of the patch synthesis technique ignores the one or more portions of the target region that are indicated as inactive by the active mask for the at least one iteration of the patch synthesis technique.

2. The one or more computer-readable storage media of claim 1 , wherein generation of an active mask further includes:
performance of a coherency analysis on individual pixels of the target region, and wherein
identification of the groupings of pixels that have converged is based, at least in part, on the coherency analysis.

3. The one or more computer-readable storage media of claim 2 , wherein identification of the groupings of pixels that have converged is further based on a determined coherency of a set of neighboring pixels of each pixel within the groupings of pixels that have converged.

4. The one or more computer-readable storage media of claim 2 , wherein
identification of groupings of pixels that have potentially converged based on the coherency analysis, wherein the identification of the groupings of pixels that have converged is further based on an erosion of the groupings of pixels that have potentially converged by a predetermined number of pixels.

5. The one or more computer-readable storage media of claim 4 , wherein the predetermined number of pixels is based on a patch size utilized in the patch synthesis technique.

6. The one or more computer-readable storage media of claim 1 , wherein to perform a patch synthesis technique includes:
performance of an approximate nearest-neighbor search process to identify the portions of the source region that are to be utilized in synthesizing the target region; and
performance of a voting process, based on the identified portions of the source region, to generate colors for individual pixels within the target region.

7. The one or more computer-readable storage media of claim 6 , wherein the at least one process of the patch synthesis technique that ignores the one or more portions indicated as inactive by the active mask includes the voting process.

8. The one or more computer-readable storage media of claim 6 , wherein the at least one iteration of the patch synthesis technique utilizes an active mask from a previous iteration of the patch synthesis technique, and wherein performance of the approximate nearest-neighbor search process ignores those pixels identified as inactive within the active mask from the previous iteration.

9. A computer-implemented method for synthesizing digital images comprising:
receiving input identifying a target region within an image wherein the target region is to be synthesized;
performing a patch synthesis technique to synthesize the target region based on portions of a source region within the image, wherein the portions of the source region are identified by the patch synthesis technique, at least one iteration of the patch synthesis technique including:
performing a coherency analysis on pixels within the target region; and
generating, based on a result of the coherency analysis, an active mask that indicates one or more portions of the target region as inactive, the one or more portions of the target region being composed of groupings of pixels that have been determined, in at least one other iteration of the patch analysis technique, to have converged, wherein at least one process of the patch synthesis technique ignores the one or more portions of the target region that are indicated as inactive by the active mask for the at least one iteration of the patch synthesis technique.

10. The computer-implemented method of claim 9 , further comprising:
identifying the groupings of pixels that have converged based on a determined coherency of a set of neighboring pixels of each pixel within the groupings of pixels.

11. The computer-implemented method of claim 9 , wherein generation of an active mask further includes:
identifying groupings of pixels that have potentially converged based on the coherency analysis; and
identifying the groupings of pixels that have converged based on an erosion of the groupings of pixels that have potentially converged by a predetermined number of pixels.

12. The computer-implemented method of claim 9 , wherein performing a patch synthesis technique includes:
performing an approximate nearest-neighbor search process to identify the portions of the source region that are to be utilized in synthesizing the target region; and
generating, based on the identified portions of the source region, colors for individual pixels within the target region through a voting process.

13. The computer-implemented method of claim 12 , wherein the at least one process of the patch synthesis technique that ignores the one or more portions indicated as inactive by the active mask includes the voting process.

14. The computer-implemented method of claim 12 , wherein the at least one iteration of the patch synthesis technique utilizes an active mask from a previous iteration of the patch synthesis technique, and wherein performing the approximate nearest-neighbor search process ignores those pixels identified as inactive within the active mask from the previous iteration.

15. A computing system for synthesizing digital images comprising:
one or more processors; and
one or more non-transitory computer-readable storage media, coupled with the one or more processors, having instructions stored thereon, which, when executed by the one or more processors, provide the computing system with a graphics editor to:
receive input identifying a target region within an image, wherein the target region is to be synthesized;

perform a patch synthesis technique to synthesize the target region based on portions of a source region within the image, wherein the portions of the source region are identified by the patch synthesis technique, at least one iteration of the patch synthesis technique including:
generation of an active mask that indicates one or more portions of the target region as inactive, the one or more portions of the target region being composed of groupings of pixels that have converged in at least one other iteration of the patch synthesis technique, wherein at least one process of the patch synthesis technique ignores the one or more portions of the target region that are indicated as inactive by the active mask for the at least one iteration of the patch synthesis technique.

16. The computing system of claim 15 , wherein generation of an active mask further includes:
performance of a coherency analysis on individual pixels of the targe

Assignees

• Adobe Systems Inc

Inventors

• Amirghodsi Sohrab
• Shechtman Elya
• Goldman Daniel Robert
• Darabi Aliakbar

Classifications

• G06T11/10

  Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) · CPC title

• G06T2200/24

  involving graphical user interfaces [GUIs] · CPC title

• G06T3/4007

  based on interpolation, e.g. bilinear interpolation (image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403) · CPC title

• G06T7/11

  Region-based segmentation · CPC title

• G06T2207/10024

  Color image · CPC title