Stereoscopic rendering using raymarching and a virtual view broadcaster for such rendering

Patent metadata

Field	Value
Publication number	US-10839591-B2
Application number	US-201815862341-A
Country	US
Kind code	B2
Filing date	Jan 4, 2018
Priority date	Jan 4, 2017
Publication date	Nov 17, 2020
Grant date	Nov 17, 2020

Abstract

The disclosure provides a virtual view broadcaster, a cloud-based renderer, and a method of providing stereoscopic images. In one embodiment, the method includes (1) generating a monoscopic set of rendered images and (2) converting the set of rendered images into a stereoscopic pair of images employing depth information from the monoscopic set of rendered images and raymarching.

First claim

What is claimed is:

1. A method of providing stereoscopic images, comprising:
generating a monoscopic cube map of rendered images, wherein the rendered images are computer-generated images; and
converting the monoscopic cube map of the rendered images into a stereoscopic pair of images by raymarching employing depth information from the monoscopic cube map of the rendered images and projecting rays on each face of the monoscopic cube map employing omni-directional stereo (ODS) projection.

2. The method as recited in claim 1 wherein the monoscopic cube map of the rendered images includes both a color buffer and a depth buffer, and the stereoscopic pair of images are a stereoscopic pair of cube maps.

3. The method as recited in claim 2 wherein the converting includes determining a color for left eye pixels and a color for a right eye pixels for single points on a face of the monoscopic cube map.

4. The method as recited in claim 3 wherein the raymarching includes determining, employing the depth buffer, surfaces on the face that intersect with the rays.

5. The method as recited in claim 4 wherein the depth buffer includes mipmap levels and the raymarching includes employing the mipmap levels to determine the surfaces on the face that intersect with the rays.

6. The method as recited in claim 3 wherein the face is one of six faces of the monoscopic cube map and the converting includes the ODS projection projecting rays on each of the six faces to determine left and right eye pixel colors for points on each of the six faces employing the raymarching.

7. The method as recited in claim 6 wherein the raymarching selects the face to start with based on an origin of one of the rays.

8. The method as recited in claim 3 further comprising reducing a diameter of a projection circle of the ODS projection to remove spiral distortion of the stereoscopic pair of images.

9. The method as recited in claim 1 further comprising rendering translucent geometry for each of the stereoscopic pair of images after the raymarching.

10. The method as recited in claim 9 further comprising reprojecting the stereoscopic pair of images for video transmission.

11. The method as recited in claim 1 wherein the monoscopic cube map of the rendered images is a 360 degree monoscopic cube map of the rendered images and the stereoscopic pair of images is a 360 degree stereoscopic pair of cube maps.

12. A virtual view broadcaster, comprising:
a cloud-based renderer configured to generate a 360 degree monoscopic cube map of rendered images and convert the 360 degree monoscopic set of rendered images into a 360 degree stereoscopic pair of images by raymarching employing depth information from the 360 degree monoscopic cube map of rendered images and projecting rays on each face of the 360 degree monoscopic cube map employing omni-directional stereo (ODS) projection, wherein the rendered images are computer-generated images; and
an image processor configured to encode the stereoscopic pair of images for video transmission.

13. The virtual view broadcaster as recited in claim 12 further comprising a video transmitter configured to transmit the 360 degree stereoscopic pair of images as a video stream over a communications network.

14. The virtual view broadcaster as recited in claim 12 wherein the 360 degree monoscopic cube map of the rendered images includes both a color buffer and a depth buffer, and the 360 degree stereoscopic pair of images are a stereoscopic pair of cube maps.

15. The virtual view broadcaster as recited in claim 14 wherein said projecting determines a color for left eye pixels and a color for right eye pixels for single points on each of the faces of the monoscopic cube map.

16. The virtual view broadcaster as recited in claim 14 wherein the raymarching includes employing the depth buffer for determining surfaces on the each face of the monoscopic cube map that intersect with projected rays.

17. The virtual view broadcaster as recited in claim 12 wherein the cloud-based renderer is further configured to render translucent geometry for each of the stereoscopic pair of images after the raymarching.

18. A cloud-based renderer, comprising:
a memory; and
at least one processor coupled to the memory and configured to convert a monoscopic cube map to a stereoscopic pair of cube maps by raymarching employing a depth buffer of the monoscopic cube map, and by projecting rays on each face of the mono scopic cube map employing omni-directional stereo (ODS) projection.

19. The cloud-based renderer as recited in claim 18 wherein projecting the rays determines a color for left eye pixels and a color for right eye pixels for single points on each of the faces of the monoscopic cube map.

20. The cloud-based renderer as recited in claim 18 wherein the processor is further configured to render translucent geometry for each of the stereoscopic pair of cube maps after the raymarching.

21. The cloud-based renderer as recited in claim 18 wherein the at least one processor is further configured to generate the monoscopic cube map.

Assignees

• Nvidia Corp

Inventors

• Panteleev Alexey
• Duka Dmitry

Classifications

• A63F13/212 Primary

  using sensors worn by the player, e.g. for measuring heart beat or leg activity · CPC title

• H04N13/257

  Colour aspects · CPC title

• H04N21/21805

  enabling multiple viewpoints, e.g. using a plurality of cameras · CPC title

• A63F2300/538

  for performing operations on behalf of the game client, e.g. rendering · CPC title

• G06T19/003

  Navigation within 3D models or images · CPC title