Electronic device for compositing images on basis of depth map and method therefor

Patent metadata

Field	Value
Publication number	US-2025069322-A1
Application number	US-202418941838-A
Country	US
Kind code	A1
Filing date	Nov 8, 2024
Priority date	May 12, 2022
Publication date	Feb 27, 2025
Grant date	—

Abstract

An electronic device according to one embodiment may include memory storing instructions and at least one processor operably coupled to the memory. The at least one processor may be configured to, when the instructions are executed, identify a first image comprising one or more areas distinguished by one or more colors; obtain at least one depth map based on the first image, wherein the at least one depth map comprises the one or more areas in the first image; and obtain, based on the first image and the at least one depth map, a virtual image including one or more subjects indicated by colors of the one or more areas.

First claim

What is claimed is:

1 . An electronic device comprising:
memory storing instructions; and at least one processor operably coupled to the memory, wherein the at least one processor is configured to: identify a first image comprising one or more areas distinguished by one or more colors; obtain at least one depth map based on the first image, wherein the at least one depth map comprises the one or more areas in the first image; and obtain, based on the first image and the at least one depth map, a virtual image including one or more subjects indicated by colors of the one or more areas.

2 . The electronic device of claim 1 , wherein the at least one depth map comprises a first depth value that is assigned to a first pixel within a first area among the one or more areas, and a second depth value that is assigned to a second pixel within the first area, wherein the first depth value is different from the second depth value, and wherein the second pixel is different from the first pixel.

3 . The electronic device of claim 1 , wherein the first image comprises a plurality of areas distinguished by a plurality of colors, and wherein the at least one processor is further configured to:
obtain, based on the first image and the at least one depth map, the virtual image, wherein the virtual image comprises a plurality of subjects having distinct types, with the plurality of subjects are respectively corresponding to the plurality of colors.

4 . The electronic device of claim 1 , wherein the at least one processor is further configured to:
obtain, based on the first image, the at least one depth map; obtain, in response to an input indicating a selection of a first depth map among the at least one depth map, the virtual image based on the first depth map and the first image.

5 . The electronic device of claim 1 , further comprises,
a display, wherein the at least one processor is further configured to: display, in response to obtaining the at least one depth map, a screen to adjust at least one depth value included in the at least one depth map, on the display.

6 . The electronic device of claim 1 , wherein the at least one processor is further configured to:
obtain the at least one depth map by inputting the first image and at least one random number to a neural network indicated by a plurality of parameters stored in the memory.

7 . The electronic device of claim 1 , wherein the at least one processor is further configured to:
obtain the virtual image by inputting the at least one depth map, the first image, and at least one random number to a neural network indicated by a plurality of parameters stored in the memory.

8 . The electronic device of claim 1 , wherein the first image is a semantic map to indicate the one or more subjects, wherein the one or more subjects are indicated based on at least one of a shape of the one or more areas, or the one or more colors which are filled in the one or more areas.

9 . The electronic device of claim 1 , wherein the virtual image includes terrain indicated by the at least one depth map.

10 . The electronic device of claim 1 , wherein the at least one processor is further configured to:
obtain, based on the first image, the at least one depth map indicating depth distribution within the one or more areas, obtain the virtual image including the one or more subjects positioned based on the depth distribution.

11 . A method of generating a virtual image, the method being executed by at least one processor of an electronic device, the method comprising:
identifying a semantic map indicating shapes and locations of one or more subjects; obtaining a plurality of candidate depth maps based on the semantic map, wherein the plurality of candidate depth maps comprise depth values of a plurality of pixels included in the semantic map; identifying a depth map corresponding to the semantic map based on the plurality of candidate depth maps; and obtaining, one or more images in which the one or more subjects are positioned based on the identified depth map, and the semantic map.

12 . The method of claim 11 , wherein the semantic map comprises:
a plurality of areas in which distinct colors are filled, wherein the distinct colors indicate types of the one or more subjects, and shapes of the plurality of areas indicate the shapes of the one or more subjects.

13 . The method of claim 12 , wherein the obtaining the plurality of candidate depth maps comprises:
obtaining the plurality of candidate depth maps based on using a neural network receiving the semantic map and at least one numeric value, wherein the plurality of candidate depth maps comprise depth distribution within a first area among the plurality of areas.

14 . The method of claim 11 , wherein the identifying the depth map comprises:
displaying the plurality of candidate depth maps on a display of the electronic device; receiving an input indicating selection of a first depth map among the plurality of candidate depth maps; and identifying the first depth map by the input, as a depth map corresponding to the semantic map.

15 . The method of claim 11 , wherein the obtaining the one or more images comprises:
obtaining, using a neural network receiving the identified depth map and one or more random numbers, the one or more images, wherein a number of the one or more images is matched to a number of the one or more random numbers.

16 . A non-transitory computer readable medium storing instructions, wherein the instructions cause at least one processor to:
identifying a first image comprising one or more areas distinguished by one or more colors; obtaining at least one depth map based on the first image, wherein the at least one depth map comprises the one or more areas included in the first image; and obtaining, based on the first image and the at least one depth map, a virtual image including one or more subjects indicated by colors of the one or more areas.

17 . The non-transitory computer readable medium of claim 16 , wherein the at least one depth map includes,
a first depth value that is assigned to a first pixel within a first area among the one or more areas, and a second depth value that is assigned to a second pixel within the first area, wherein the first depth value is different from the second depth value, and wherein the second pixel is different from the first pixel.

18 . The non-transitory computer readable medium of claim 16 , wherein the first image comprises a plurality of areas distinguished by a plurality of colors, and wherein the obtaining the virtual image comprises:
obtaining, based on the first image and the at least one depth map, wherein the virtual image comprises a plurality of subjects having distinct types, with the plurality of subjects are respectively corresponding to the plurality of colors.

19 . The non-transitory computer readable medium of claim 16 , wherein the obtaining the at least one depth map comprises:
obtaining, based on the first image, the at least one depth map, wherein the obtaining the virtual image comprises: obtaining, in response to an input indicating a selection of a first depth map among the at least one depth maps, the virtual image based on the first depth map, and the first image.

20 . The non-transitory computer readable medium of claim 16 , wherein the obtaining the at least one depth map comprises:
displaying, in response to obtaining the at least one depth map, a screen to adjust at le

Assignees

• Ncsoft Corp

Inventors

• Lee Gunhee
• Yim Jonghwa
• Kim Chanran
• Kim Minjae

Classifications

• G06T2207/10024

  Color image · CPC title

• G06T15/205 Primary

  Image-based rendering · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T7/50 Primary

  Depth or shape recovery · CPC title

• G06T2200/24

  involving graphical user interfaces [GUIs] · CPC title