Devices, systems, and methods for generating multi-modal images of a synthetic scene

Patent metadata

Field	Value
Publication number	US-10277859-B2
Application number	US-201715494352-A
Country	US
Kind code	B2
Filing date	Apr 21, 2017
Priority date	Sep 14, 2016
Publication date	Apr 30, 2019
Grant date	Apr 30, 2019

Abstract

Devices, systems, and methods obtain an object model, add the object model to a synthetic scene, add a texture to the object model, add a background plane to the synthetic scene, add a support plane to the synthetic scene, add a background image to one or both of the background plane and the support plane, and generate a pair of images based on the synthetic scene, wherein a first image in the pair of images is a depth image of the synthetic scene, and wherein a second image in the pair of images is a color image of the synthetic scene.

First claim

What is claimed is:

1. A system comprising:
one or more non-transitory computer-readable media; and
one or more processors that are coupled to the one or more computer-readable media and that are configured to cause the system to
obtain an object model;
add the object model to a synthetic scene;
add a texture to the object model;
add a background plane to the synthetic scene;
add a support plane to the synthetic scene;
add a background image to one or both of the background plane and the support plane; and
generate a pair of images based on the synthetic scene, wherein a first image in the pair of images is a depth image of the synthetic scene, and wherein a second image in the pair of images is a color image of the synthetic scene.

2. The system of claim 1 , wherein the one or more processors are further configured to cause the system to
select a position of a simulated image sensor, wherein the first image and the second image are generated from a perspective of the simulated image sensor.

3. The system of claim 2 , wherein the one or more processors are further configured to cause the system to
select, at random, a pose of the object model relative to the simulated image sensor.

4. The system of claim 1 , wherein the one or more processors are further configured to cause the system to deform the background plane.

5. The system of claim 4 , wherein, to deform the background plane, the one or more processors are further configured to cause the system to
add extrusions to the background plane,
add intrusions to the background plane, or
add noise to the background plane.

6. The system of claim 1 , wherein the one or more processors are further configured to cause the system to
test different sizes of the object model for compatibility with a scale of the synthetic scene;
select one of the different sizes of the object model; and
set a size of the object model to the selected one of the different sizes.

7. The system of claim 1 , wherein the one or more processors are further configured to cause the system to
generate an altered synthetic scene by performing one or more of the following:
adding a different texture to the object model,
adding a different background image to one or both of the background plane and the support plane,
changing a size of the object model,
changing a position of a simulated image sensor, and
changing a pose of the object model; and

generate a second pair of images based on the altered synthetic scene, wherein a first image in the second pair of images is a depth image of the altered synthetic scene, and wherein a second image in the second pair of images is a color image of the altered synthetic scene.

8. One or more non-transitory computer-readable storage media storing computer-executable instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations comprising:
adding an object model to a synthetic scene;
adding a texture to the object model;
adding a background plane to the synthetic scene;
adding a background image to the background plane; and
generating a pair of images based on the synthetic scene, wherein a first image in the pair of images is a depth image of the synthetic scene, and wherein a second image in the pair of images is an illumination-map image of the synthetic scene.

9. The one or more computer-readable storage media of claim 8 , wherein the operations further comprise:
adding a support plane to the synthetic scene.

10. The one or more computer-readable storage media of claim 9 , wherein the support plane is added to the synthetic scene below the object model.

11. The one or more computer-readable storage media of claim 9 , wherein the support plane is added to the synthetic scene above the object model.

12. The one or more computer-readable storage media of claim 8 , wherein the object model is a computer-aided-design (CAD) model.

13. The one or more computer-readable storage media of claim 8 , wherein the operations further comprise
determining a position of a simulated image sensor in the synthetic scene, wherein the pair of images is generated from the perspective of the simulated image sensor.

14. The one or more computer-readable storage media of claim 13 , wherein the operations further comprise:
positioning the object model and the background plane in the synthetic scene such that both the object model and the background plane are within a depth range of the simulated image sensor.

15. A method comprising:
selecting an object model from a first object category;
adding the object model to a synthetic scene;
selecting a texture from a first texture category, wherein the first texture category corresponds to the first object category;
adding the texture to the object model;
adding a background plane to the synthetic scene;
selecting a background image from a first background-image category, wherein the first background-image category corresponds to the first object category;
adding the background image to the background plane; and
generating a pair of images based on the synthetic scene, wherein a first image in the pair of images is a depth image of the synthetic scene, and wherein a second image in the pair of images is an illumination-map image of the synthetic scene.

16. The method of claim 15 , wherein the texture is an image of wood, metal, ceramic, or textile.

17. The method of claim 15 , wherein the first object category is furniture, wherein the first texture category is materials that compose furniture, and wherein the first background-image category is scenes that include furniture.

18. The method of claim 15 , further comprising:
adding a support plane to the synthetic scene.

19. The method of claim 18 , further comprising:
adding the background image to the support plane.

20. The method of claim 18 , further comprising:
adding a second background image to the support plane, wherein the second background image is different from the background image.

Assignees

• Canon Kk

Inventors

• Armeni Iro
• Skaff Sandra
• Yu Jie

Classifications

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T2207/10024

  Color image · CPC title

• H04N5/44504 Primary

  Circuit details of the additional information generator, e.g. details of the character or graphics signal generator, overlay mixing circuits · CPC title

• H04N21/845

  Structuring of content, e.g. decomposing content into time segments · CPC title