Synthesizing cloud stickers

Patent metadata

Field	Value
Publication number	US-11715242-B2
Application number	US-202117448139-A
Country	US
Kind code	B2
Filing date	Sep 20, 2021
Priority date	Nov 21, 2017
Publication date	Aug 1, 2023
Grant date	Aug 1, 2023

Abstract

Disclosed are systems, methods, and computer-readable storage media to modify image content. One aspect includes identifying, by one or more electronic hardware processors, an image and content within the image, determining, by the one or more electronic hardware processors, a sky region of the image, determining, by the one or more electronic hardware processors, whether the content within the image is located within the sky region of the image, and in response to the content being within the sky region of the image, modifying, by the one or more electronic hardware processors, the content based on fractal Brownian motion.

First claim

We claim:

1. A method comprising:
determining, by one or more electronic hardware processors, placement of an item within an image;
displaying the image, by the one or more electronic hardware processors; and
modifying, by the one or more electronic hardware processors, the item to have a texture resembling a water region or a sky region of the image, the item being modified to have the texture resembling an attribute of the sky region in response to determining that the item was placed within the sky region.

2. The method of claim 1 , further comprising modifying the item to have the texture resembling an attribute of the water region.

3. The method of claim 1 , wherein the item comprises a sticker, and further comprising:
invoking a classifier trained to segment an image into the sky region and a non-sky region.

4. The method of claim 1 , wherein prior to modifying the item, determining that the item has dissimilar visual properties than the texture of the water or sky region, and wherein modifying the item comprises generating a cloud-like texture based on fractal Brownian motion.

5. The method of claim 1 , wherein modifying the item further comprises:
generating a grayscale version of the item;
blurring the grayscale version of the item; and
multiplying pixels of the blurred version of the item and pixels of a cloud-like texture to modify the item to have the cloud-like texture.

6. The method of claim 1 , further comprising refraining from modifying the item in response to the item being outside a specified region of the image.

7. The method of claim 1 , wherein the image is defined by image data and the item is defined by item data separate from the image data, and wherein the item is a colored emoji.

8. The method of claim 1 , further comprising:
overlaying the modified item on the image at a determined location based on item location information.

9. The method of claim 1 , further comprising:
determining a specified region of the image, the item being placed within the specified region of the image.

10. The method of claim 9 , wherein the item is modified in response to determining placement of the item within the specified region of the image, the item being modified based on an attribute of the specified region.

11. The method of claim 9 , wherein the texture of the item is modified to resemble a texture of the specified region.

12. An apparatus comprising:
one or more hardware processors; and
a hardware memory, operably connected to the one or more hardware processors, and storing instructions that when executed by one or more hardware processors, perform operations comprising:
determining placement of an item within an image;
modifying the item to have a texture resembling a water region or a sky region of the image; and

displaying the image, the item being modified to have the texture resembling an attribute of the sky region in response to determining that the item was placed within the sky region.

13. The apparatus of claim 12 , wherein modifying the item further comprises:
modifying the item to have the texture resembling an attribute of the water region.

14. The apparatus of claim 12 , further comprising operations for:
invoking a classifier trained to segment an image into the sky region and a non-sky region.

15. The apparatus of claim 12 , wherein prior to modifying the item, the item has a different visual property and texture than the texture of the water or sky region, and wherein the operations for modifying the item comprise operations for generating a cloud-like texture based on fractal Brownian motion.

16. The apparatus of claim 12 , wherein the operations for modifying the item further comprise operations for:
generating a grayscale version of the item;
blurring the grayscale version of the item; and
multiplying pixels of the blurred version of the item and pixels of a cloud-like texture to modify the item to have the cloud-like texture.

17. The apparatus of claim 12 , further comprising operations for:
determining a specified region of the image, the item being placed within the specified region of the image, wherein the item is modified in response to determining placement of the item within the specified region of the image, the item being modified based on an attribute of the specified region, and wherein the texture of the item is modified to resemble a texture of the specified region.

18. A non-transitory computer-readable medium comprising non-transitory computer readable instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
determining placement of an item within an image;
modifying the item to have a texture resembling a water region or a sky region of the image; and
displaying the image, the item being modified to have the texture resembling an attribute of the sky region in response to determining that the item was placed within the sky region.

19. The non-transitory computer-readable medium of claim 18 , wherein prior to modifying the item, the operations comprising determining that the item has dissimilar visual properties than the texture of the water or sky region, and wherein modifying the item comprises generating a cloud-like texture based on fractal Brownian motion.

20. The non-transitory computer-readable medium of claim 18 , the operations comprising invoking a classifier trained to segment an image into the sky region and a non-sky region.

Assignees

• Snap Inc

Inventors

• Ma Chongyang
• Mei Xing
• Hu Nan
• Ouimet Kirk

Classifications

• G06T11/10 Primary

  Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) · CPC title

• G06T11/001 Primary

  Physics · mapped topic

• G06T5/002

  Physics · mapped topic

• G06T7/11

  Region-based segmentation · CPC title

• G06T7/277

  involving stochastic approaches, e.g. using Kalman filters · CPC title