Experience learning in virtual world

Patent metadata

Field	Value
Publication number	US-2020356899-A1
Application number	US-202016868311-A
Country	US
Kind code	A1
Filing date	May 6, 2020
Priority date	May 6, 2019
Publication date	Nov 12, 2020
Grant date	—

Abstract

A computer-implemented method of machine-learning is described that includes obtaining a test dataset of scenes. The test dataset belongs to a test domain. The method includes obtaining a domain-adaptive neural network. The domain-adaptive neural network is a machine-learned neural network taught using data obtained from a training domain. The domain-adaptive neural network is configured for inference of spatially reconfigurable objects in a scene of the test domain. The method further includes determining an intermediary domain. The intermediary domain is closer to the training domain than the test domain in terms of data distributions. The method further includes inferring, by applying the domain-adaptive neural network, a spatially reconfigurable object from a scene of the test domain transferred on the intermediary domain. Such a method constitutes an improved method of machine learning with a dataset of scenes comprising spatially reconfigurable objects.

First claim

1 . A computer-implemented method of machine-learning, comprising:
obtaining:
a test dataset of scenes belonging to a test domain, and
a domain-adaptive neural network, the domain-adaptive neural network being a machine-learned neural network taught using data obtained from a training domain, the domain-adaptive neural network being configured for inference of spatially reconfigurable objects in a scene of the test domain;
determining an intermediary domain, the Intermediary domain being closer to the training domain than the test domain in terms of data distributions; and inferring, by applying the domain-adaptive neural network, a spatially reconfigurable object from a scene of the test domain transferred on the intermediary domain.

2 . The method of claim 1 , wherein the determining of the intermediary domain includes, for each scene of the test dataset:
transforming the scene of the test dataset into another scene closer to the training domain in terms of data distribution.

3 . The method of claim 2 , wherein the transforming of the scene includes:
generating a virtual scene from the scene of the test dataset, the another scene being inferred based on the scene of the test dataset and on the generated virtual scene.

4 . The method of claim 3 , wherein the generating of the virtual scene from the scene of the test dataset Includes applying a virtual scene generator to the scene of the test dataset, the virtual scene generator being a machine-learned neural network configured for Inference of virtual scenes from scenes of the test domain.

5 . The method of claim 4 , wherein the virtual scene generator has been taught used a dataset of scenes each including one or more spatially reconfigurable objects.

6 . The method of claim 3 , wherein the determining of the intermediary domain further includes blending the scene of the test dataset and the generated virtual scene, the blending resulting in the another scene.

7 . The method of claim 6 , wherein the blending of the scene of the test dataset and of the generated virtual scene is a linear blending.

8 . The method of claim 1 , wherein the test dataset includes real scenes.

9 . The method of claim 8 , wherein each real scene of the test dataset is a real manufacturing scene Includes one or more spatially reconfigurable manufacturing tools, the domain-adaptive neural network being configured for inference of spatially reconfigurable manufacturing tools in a real manufacturing scene.

10 . The method of claim 1 , wherein the training domain includes a training dataset of virtual scenes each Including one or more spatially reconfigurable objects.

11 . The method of claim 10 , wherein each virtual scene of the dataset of virtual scenes is a virtual manufacturing scene including one or more spatially reconfigurable manufacturing tools, the domain-adaptive neural network being configured for inference of spatially reconfigurable manufacturing tools in a real manufacturing scene.

12 . The method of claim 1 , wherein the data obtained from the training domain includes scenes of another intermediary domain, the domain-adaptive neural network having been taught using the another intermediary domain, the another intermediary domain being closer to the intermediary domain than the training domain in terms of data distributions.

13 . A device comprising:
a non-transitory data storage medium having recorded thereon a computer program including instructions for machine-learning that when executed by a processor causes the processor to be configured to obtain:
a test dataset of scenes belonging to a test domain, and
a domain-adaptive neural network, the domain-adaptive neural network being a machine-learned neural network taught using data obtained from a training domain, the domain-adaptive neural network being configured for inference of spatially reconfigurable objects in a scene of the test domain;
determine an Intermediary domain, the Intermediary domain being closer to the training domain than the test domain in terms of data distributions, and infer, by applying the domain-adaptive neural network, a spatially reconfigurable object from a scene of the test domain transferred on the intermediary domain.

14 . The device of claim 13 , wherein the processor is further configured to determine the intermediary domain by being further configured to, for each scene of the test dataset:
transform the scene of the test dataset into another scene closer to the training domain in terms of data distribution.

15 . The device of claim 14 , wherein the processor is further configured to transform of the scene by being further configured to
generate a virtual scene from the scene of the test dataset, the another scene being Inferred based on the scene of the test dataset and on the generated virtual scene.

16 . The device of claim 15 , wherein the processor is further configured to generate of the virtual scene from the scene of the test dataset by being further configured to apply a virtual scene generator to the scene of the test dataset, the virtual scene generator being a machine-learned neural network configured for inference of virtual scenes from scenes of the test domain.

17 . The device of claim 16 , wherein the virtual scene generator has been taught using a dataset of scenes each including one or more spatially reconfigurable objects.

18 . The device of claim 15 , wherein the processor is further configured to determine of the intermediary domain by being further configured to blend the scene of the test dataset and the generated virtual scene, the blending resulting in the another scene.

19 . The device of claim 18 , wherein the blending of the scene of the test dataset and of the generated virtual scene is a linear blending.

20 . The device of claim 13 , further comprising the processor coupled to the non-transitory data storage medium.

Assignees

• Dassault Systemes

Inventors

• Rejeb Sfar Asma
• Boulkenafed Malika
• Mezghanni Mariem

Classifications

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title

• G06N3/08 Primary

  Learning methods · CPC title

• G06F18/25

  Fusion techniques · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/094

  Adversarial learning · CPC title