Automated generation of synthetic lighting scene images using generative adversarial networks

Patent metadata

Field	Value
Publication number	US-2020380652-A1
Application number	US-202016881066-A
Country	US
Kind code	A1
Filing date	May 22, 2020
Priority date	May 30, 2019
Publication date	Dec 3, 2020
Grant date	—

Abstract

This disclosure is directed to systems and methods for automated generation of lighting scene images. An image of an environment is provided to the system, and lamps with particular light styles can be added to various locations on the image. A modified image of the environment which includes the added lamps and light styles is generated using a generative adversarial network. The generative adversarial network focuses on one or more zones around the added lamp and applies learned or pre-specified decay functions to the light style in the zones.

First claim

1 . A method for automated generation of lighting scene images, comprising:
receiving an image of an environment; receiving a location on the image of an environment of a first lamp or obtaining a learned location on the image of an environment of the first lamp; identifying one or more zones at varied distances from the first added lamp; inserting an image of the first lamp on the image of an environment; identifying one or more spatial light style decay functions suitable to the image of an environment based on the first added lamp; and generating, using a conditional generative adversarial network, a modified image of an environment, wherein the modified image shows the first added lamp and shows one or more light styles from the first added lamp, the one or more light styles being based on the identified one or more spatial light style decay functions, and associated with the one or more identified zones.

2 . The method of claim 1 , further comprising:
receiving or learning one or more interaction constraints regarding light style.

3 . The method of claim 1 , wherein the received or learned location on the image of the environment to insert the first lamp is determined by a modified object detection network or a regional proposal segmentation based neural network.

4 . The method of claim 1 , further comprising the step of transforming the modified image using the conditional generative adversarial network or style transfer network, wherein transforming the modified image comprises at least one of: rotating the modified image; altering a view point of the modified image; and
changing ambient background conditions.

5 . The method of claim 1 , wherein the conditional generative adversarial network comprises one or more generator models and one or more discriminator models.

6 . The method of claim 1 , wherein the one or more light styles are regarding at least one of: color, brightness, spectrum, shadow, reflectivity, hue, and light source location.

7 . The method of claim 1 , wherein the conditional generative adversarial network is trained using image pairs where one image of the image pairs shows an environment with in-place lamps and another image of the image pairs shows an environment without in-place lamps.

8 . The method of claim 1 , wherein the conditional generative adversarial network is trained using images with one or more in-place lamps, further comprising the steps of:
running a detection model to detect a mask of the one or more in-place lamps and lamp styles of the one or more in-place lamps; removing the mask of the one or more in-place lamps; and removing one or more in-place lamp styles using image decomposition.

9 . A system for automated generation of lighting scene images, comprising:
a first computing device, having a first communication module, a first memory, and a first processor, wherein the first processor is configured to:
send, to a second computing device via the first communication module, an image of an environment; and
the second computing device, having a second communication module, a second memory, and a second processor, wherein the second processor is configured to: receive, from the first computing device via the second communication module, an image of an environment; receive a location on the image of an environment of a first lamp or obtain a learned location on the image of an environment of a first lamp, from the first computing device or the second computing device; identify one or more zones at varied distances from the first lamp; insert an image of the first lamp on the image of an environment; receive or learn one or more spatial light style decay functions; and generate, using a conditional generative adversarial network, a modified image of an environment, wherein the modified image shows the first added lamp and shows one or more light styles from the first added lamp, the one or more light styles being based on the one or more spatial light style decay functions, and associated with the one or more identified zones.

10 . The system of claim 9 , wherein the second computing device is further configured to:
receive or learn one or more interaction constraints regarding light style.

11 . The system of claim 9 , wherein the first processor of the first computing device is further configured to: send, to the second computing device via the first communication module, one or more light styles regarding the first added lamp.

12 . The system of claim 9 , wherein the second processor of the second computing device is further configured to: transform the modified image using the conditional generative adversarial network or style transfer network, wherein transforming the modified image comprises at least one of: rotating the modified image; altering a view point of the modified image; and changing ambient background conditions.

13 . The system of claim 9 , wherein the conditional generative adversarial network comprises one or more generator models and one or more discriminator models.

14 . The system of claim 9 , wherein the one or more light styles are regarding at least one of color, brightness, spectrum, shadow, reflectivity, hue, and light source location.

Assignees

• Signify Holding Bv

Inventors

• Olaleye Olaitan Philip
• Gao Jian

Classifications

• G06T11/10

  Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) · CPC title

• G06T11/00 Primary

  Two-dimensional [2D] image generation · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06V10/774

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V10/82

  using neural networks · CPC title