Image generation using one or more neural networks

Patent metadata

Field	Value
Publication number	US-2021097691-A1
Application number	US-201916588910-A
Country	US
Kind code	A1
Filing date	Sep 30, 2019
Priority date	Sep 30, 2019
Publication date	Apr 1, 2021
Grant date	—

Abstract

Apparatuses, systems, and techniques are presented to generate or manipulate digital images. In at least one embodiment, a network is trained to generate modified images including user-selected features.

First claim

What is claimed is:

1 . A processor, comprising:
one or more circuits to help generate a modified image, including one or more second features selected by a user to replace one or more first features within an original image.

2 . The processor of claim 1 , wherein the original image is an image captured using a camera or an image generated from an initial segmentation mask.

3 . The processor of claim 1 , wherein the one or more circuits are to apply one or more style filters to content to be rendered for at least a subset of the one or more second features.

4 . The processor of claim 1 , wherein the one or more circuits are further to determine one or more segmentation boundaries for the one or more features of the original image, and wherein the segmentation boundaries are enabled to be added, deleted, or modified.

5 . The processor of claim 4 , wherein deletion of a segmentation boundary enables removal of an object represented in the original image.

6 . The processor of claim 1 , wherein the one or more circuits are to use one or more neural networks to determine a different type of content to be rendered in at least one of the one or more second features with respect to the original image.

7 . The processor of claim 1 , wherein the modified image is of a higher resolution than the original image.

8 . A system, comprising:
one or more processors to generate a modified image, including one or more second features selected by a user to replace one or more first features within an original image; and memory to store the modified image.

9 . The system of claim 8 , wherein the original image is an image captured using a camera or an image generated from an initial segmentation mask.

10 . The system of claim 8 , wherein the one or more circuits are to apply one or more style filters to content to be rendered for at least a subset of the one or more second features.

11 . The system of claim 8 , wherein the one or more circuits are further to determine one or more segmentation boundaries for the one or more features of the original image, and wherein the segmentation boundaries are enabled to be added, deleted, or modified.

12 . The system of claim 11 , wherein deletion of a segmentation boundary enables removal of an object represented in the original image.

13 . The system of claim 8 , wherein the one or more circuits are to use one or more neural networks to determine a different type of content to be rendered in at least one of the one or more second features with respect to the original image.

14 . The system of claim 8 , wherein the modified image is of a higher resolution than the original image.

15 . A method, comprising:
generating a modified image, including one or more second features selected by a user to replace one or more first features within an original image.

16 . The method of claim 15 , wherein the original image is an image captured using a camera or an image generated from an initial segmentation mask.

17 . The method of claim 15 , further comprising:
applying one or more style filters to content to be rendered for at least a subset of the one or more second features.

18 . The method of claim 15 , further comprising:
enabling one or more segmentation boundaries determined for the one or more features of the original image to be added, deleted, or modified.

19 . The method of claim 18 , wherein deletion of a segmentation boundary enables removal of an object represented in the original image.

20 . The method of claim 15 , wherein the one or more circuits are to use one or more neural networks to determine a different type of content to be rendered in at least one of the one or more second features with respect to the original image.

21 . The method of claim 15 , wherein the modified image is of a higher resolution than the original image.

22 . A machine-readable medium having stored thereon a set of instructions, which if performed by one or more processors, cause the one or more processors to at least:
generate a modified image, including one or more second features selected by a user to replace one or more first features within an original image.

23 . The machine-readable medium of claim 22 , wherein the original image is an image captured using a camera or an image generated from an initial segmentation mask.

24 . The machine-readable medium of claim 22 , wherein the one or more circuits are to apply one or more style filters to content to be rendered for at least a subset of the one or more second features.

25 . The machine-readable medium of claim 22 , wherein the one or more circuits are further to determine one or more segmentation boundaries for the one or more features of the original image, and wherein the segmentation boundaries are enabled to be added, deleted, or modified.

26 . The machine-readable medium of claim 25 , wherein deletion of a segmentation boundary enables removal of an object represented in the original image.

27 . The machine-readable medium of claim 22 , wherein the one or more circuits are to use one or more neural networks to determine a different type of content to be rendered in at least one of the one or more second features with respect to the original image.

28 . The machine-readable medium of claim 22 , wherein the modified image is of a higher resolution than the original image.

29 . A processor comprising:
one or more circuits to train one or more neural networks to generate a modified image, including one or more second features selected by a user to replace one or more first features within an original image.

30 . The processor of claim 29 , wherein the original image is an image captured using a camera or an image generated from an initial segmentation mask.

31 . The processor of claim 29 , wherein the one or more circuits are to apply one or more style filters to content to be rendered for at least a subset of the one or more second features.

32 . The processor of claim 29 , wherein the one or more circuits are further to determine one or more segmentation boundaries for the one or more features of the original image, and wherein the segmentation boundaries are enabled to be added, deleted, or modified.

33 . The processor of claim 32 , wherein deletion of a segmentation boundary enables removal of an object represented in the original image.

34 . The processor of claim 29 , wherein the one or more circuits are to use one or more neural networks to determine a different type of content to be rendered in at least one of the one or more second features with respect to the original image.

35 . The processor of claim 29 , wherein the modified image is of a higher resolution than the original image.

Assignees

• Nvidia Corp

Inventors

• Liu Ming-Yu

Classifications

• G06T11/10 Primary

  Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) · CPC title

• G06V10/774

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06T7/11 Primary

  Region-based segmentation · CPC title