Hybrid streaming

Patent metadata

Field	Value
Publication number	US-10347040-B2
Application number	US-201615387189-A
Country	US
Kind code	B2
Filing date	Dec 21, 2016
Priority date	Dec 22, 2015
Publication date	Jul 9, 2019
Grant date	Jul 9, 2019

Abstract

The invention notably relates to a computer-implemented method for displaying a 3D assembly of modeled objects. The method comprises streaming from a first computer to a second computer at least one raster image of a first 3D modeled object, and rendering on the second computer the 3D assembly of modeled objects by merging a second 3D modeled object with the streamed at least one raster image.

First claim

The invention claimed is:

1. A computer-implemented method for displaying a 3D assembly of modeled objects, comprising:
streaming from a first computer communicatively coupled to a second computer at least one raster image of a first 3D modeled object and a first depth buffer of the at least one raster image;
merging a second 3D modeled object with the streamed at least one raster image, the merging comprising performing a pixel-by-pixel comparison to determine, based on the first depth buffer, which fragment of the second 3D modeled object is rendered;
receiving, on the second computer, first light sources associated with the at least one raster image of the first 3D modeled object and second light sources associated with the second 3D modeled object;
computing a normal map that stores, for each pixel of the 3D assembly of modeled objects, a surface normal direction;
computing lighting effects on the 3D assembly of modeled objects as post processing starting from a surface normal direction and a material property of each pixel; and
rendering, on the second computer, the 3D assembly of modeled objects, wherein
the normal map is computed from a second depth buffer obtained by merging the second 3D modeled object with a first depth buffer of the at least one raster image.

2. The computer-implemented method of claim 1 ,
wherein streaming from the first computer to the second computer comprises simultaneously streaming from the first computer to the second computer the at least one raster image of the first 3D modeled object together with the first depth buffer of the at least one raster image.

3. The computer-implemented method of claim 2 , further comprising, before the streaming:
encoding both the at least one raster image of the first 3D modeled object and the first depth buffer in a single stream.

4. The computer-implemented method of claim 3 , wherein the encoding is performed by using a video compression scheme.

5. The computer-implemented method of claim 1 , further comprising, after receiving the first and second light sources and before computing the lighting effects:
computing an identification buffer that identifies, for each pixel of the 3D assembly of modeled objects, whether a pixel comes from the streamed at least one raster image of the first 3D modeled object or from the second 3D modeled object merged with the streamed at least one raster image.

6. The computer-implemented method of claim 1 , wherein computing the normal map from the second depth buffer comprises applying a convolution filter to the second depth buffer.

7. The computer-implemented method of claim 1 , wherein the material property of each pixel is inferred from the color of each pixel of the merged second 3D modeled object with at least one raster image.

8. The computer-implemented method of claim 1 , further comprising:
sending from the second computer to the first computer coordinates of a new viewpoint on the 3D assembly of modeled objects rendered on the second computer;
computing at least one new raster image of the first 3D modeled object according to the new viewpoint; and
streaming from the first computer to the second computer the at least one new raster image of the first 3D modeled object.

9. The computer-implemented method of claim 1 , wherein the second 3D modeled merged with the streamed at least one raster image is sent from a third computer to the second computer.

10. A computer program stored on a non-transitory computer readable medium comprising instructions for performing the method of claim 1 .

11. The computer-implemented method of claim 1 , further comprising:
sending from the second computer to the first computer coordinates of a new viewpoint on the 3D assembly of modeled objects rendered on the second computer;
computing at least one new raster image of the first 3D modeled object according to the new viewpoint; and
streaming from the first computer to the second computer the at least one new raster image of the first 3D modeled object, wherein
the second 3D modeled merged with the streamed at least one raster image is sent from a third computer to the second computer.

12. A non-transitory computer readable medium having recorded thereon a computer program that when executed by a computer causes the computer to implement a computer-implemented method for displaying a 3D assembly of modeled objects, the method comprising:
receiving from a first computer communicatively coupled to a second computer at least one raster image of a first 3D modeled object and a first depth buffer of the at least one raster image;
merging a second 3D modeled object with the received at least one raster image, the merging comprising performing a pixel-by-pixel comparison to determine, based on the first depth buffer, which fragment of the second 3D modeled object is rendered;
receiving first light sources associated with the at least one raster image of the first 3D modeled object and second light sources associated with the second 3D modeled object;
computing a normal map that stores, for each pixel of the 3D assembly of modeled objects, a surface normal direction;
computing lighting effects on the 3D assembly of modeled objects as post processing starting from a surface normal direction and a material property of each pixel; and
rendering the 3D assembly of modeled objects, wherein
the normal map is computed from a second depth buffer obtained by merging the second 3D modeled object with a first depth buffer of the at least one raster image.

13. A system comprising:
a first computer; and
a second computer,
wherein the first computer and the second computer are connected with a streaming channel,
wherein the first and second computers comprise processing circuitry coupled to a memory,
wherein the second computer further comprises a graphical user interface,
wherein each memory of the first and second computer has recorded thereon a computer program for performing displaying of a 3D assembly of modeled objects, and
wherein the processing circuitry is configured by computer programs to
stream from the first computer to the second computer at least one raster image of a first 3D modeled object and a first depth buffer of the at least one raster image,
merge a second 3D modeled object with the streamed at least one raster image, the merging comprising performing a pixel-by-pixel comparison to determine, based on the first depth buffer, which fragment of the second 3D modeled object is rendered,
receive, on the second computer, first light sources associated with the at least one raster image of the first 3D modeled object and second light sources associated with the second 3D modeled object,
compute a normal map that stores, for each pixel of the 3D assembly of modeled objects, a surface normal direction,
compute lighting effects on the 3D assembly of modeled objects as post processing starting from a surface normal direction and a material property of each pixel, and
render, on the second computer, the 3D assembly of modeled objects, wherein

the normal map is computed from a second depth buffer obtained by merging the second 3D modeled object with a first depth buffer of the at least one raster image.

Assignees

• Dassault Systemes

Inventors

• Boulkenafed Malika
• Tuffreau Jean Julien

Classifications

• G06T15/50 Primary

  Lighting effects · CPC title

• G06T17/00 Primary

  Three-dimensional [3D] modelling for computer graphics · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06F30/20

  Design optimisation, verification or simulation (optimisation, verification or simulation of circuit designs G06F30/30) · CPC title

• G06F2111/20

  Configuration CAD, e.g. designing by assembling or positioning modules selected from libraries of predesigned modules · CPC title