Finite aperture omni-directional stereo light transport

Patent metadata

Field	Value
Publication number	US-10748333-B2
Application number	US-201815881554-A
Country	US
Kind code	B2
Filing date	Jan 26, 2018
Priority date	Mar 23, 2017
Publication date	Aug 18, 2020
Grant date	Aug 18, 2020

Abstract

In various embodiments, a finite aperture omni-directional camera is modeled by aligning a finite aperture lens and focal point with the omni-directional part of the projection. For example, each point on an image plane maps to a direction in camera space. For a spherical projection, the lens can be orientated along this direction and the focal point is picked along this direction at focal distance from the lens. For a cylindrical projection, the lens can be oriented along the projected direction on the two dimensional (2D) xz-plane, as the projection is not omni-directional in the y direction. The focal point is picked along the (unprojected) direction so its projection on the xz-plane is at focal distance from the lens. The final outgoing ray can be constructed by sampling of point on this oriented lens and shooting a ray from there through the focal point.

First claim

What is claimed is:

1. A computer-implemented method comprising:
ray tracing, within a virtualized three dimensional (3D) space, from a light source or camera to a point corresponding to an image backplate that is within the virtualized 3D space and is representative of a view of an environment captured by an omni-directional stereo camera;
simulating a finite aperture with respect to a virtual omni-directional stereo camera capturing a two dimensional (2D) projection of a 3D scene by determining a projection of a ray from the point through a focal point of the virtual omni-directional stereo camera to a lens point within the finite aperture of the virtual omni-directional stereo camera;
performing a first inverse mapping to obtain an orientation in camera space that corresponds to the point and the lens point by aligning the lens and the focal point with an omni-directional part of the projection;
based on the orientation, performing a second inverse mapping to obtain a pixel on a capture plane of the virtual omni-directional stereo camera, the pixel corresponding to the point and the lens point;
looking up a color value within the image backplate that corresponds to the pixel; and
computing a ray tracing value at the point using the color value.

2. The method of claim 1 , wherein the omni-directional stereo camera comprises a cylindrical virtual reality camera and the projection comprises a cylindrical projection.

3. The method of claim 1 , wherein the projection comprises a cylindrical projection and the orientation comprises a camera angle within an xz-plane of the virtualized 3D space associated with the point and a height in a y coordinate in the camera space associated with the point.

4. The method of claim 1 , wherein the omni-directional stereo camera comprises a spherical virtual reality camera and the projection comprises a spherical projection.

5. The method of claim 1 , wherein the projection comprises a spherical projection and the orientation comprises a camera angle within an xz-plane of the virtualized 3D space associated with the point and a tilt angle of the virtual omni-directional stereo camera with respect to a y-axis the virtualized 3D space associated with the point.

6. The method of claim 1 , wherein the lens point is a randomly sampled point on the lens.

7. The method of claim 1 , wherein the ray tracing comprises determining a path between the light source or camera to the point based on identifying interactions of rays within the virtualized 3D space.

8. The method of claim 1 , further comprising rendering an image of the 2D projection of the 3D scene using the ray tracing value.

9. A system comprising:
one or more processing devices;
one or more memory devices communicatively coupled to the one or more processing devices storing programmed instructions thereon, which when executed by the one or more processing devices causes the one or more processing devices to perform a method comprising:
ray tracing, within a virtualized three dimensional (3D) space, from a light source or camera to a point corresponding to an image backplate that is within the virtualized 3D space and is representative of a stereoscopic panorama of an environment;
simulating a finite aperture with respect to a virtual panoramic stereo camera capturing a two dimensional (2D) projection of a 3D scene by determining a projection of a ray from the point through a focal point of the virtual panoramic stereo camera to a lens point within the finite aperture of the virtual panoramic stereo camera;
performing a first inverse mapping to obtain an orientation in camera space that corresponds to the point and the lens point by aligning the lens and the focal point with a stereo panoramic part of the projection;
based on the orientation, performing a second inverse mapping to obtain a pixel on a capture plane of the virtual panoramic stereo camera, the pixel corresponding to the point and the lens point;
looking up a color value within the image backplate that corresponds to the pixel; and
computing a ray tracing value at the point using the color value.

10. The system of claim 9 , wherein the stereoscopic panorama is captured using an omni-directional stereo camera.

11. The system of claim 9 , wherein the projection comprises a cylindrical projection and the orientation comprises a camera angle within an xz-plane of the virtualized 3D space associated with the point and a height in a y coordinate in the camera space associated with the point.

12. The system of claim 9 , wherein the projection comprises a spherical projection and the orientation comprises a camera angle within an xz-plane of the virtualized 3D space associated with the point and a tilt angle of the virtual panoramic stereo camera with respect to a y-axis the virtualized 3D space associated with the point.

13. The system of claim 9 , wherein the ray tracing comprises determining a path between the light source or camera to the point based on identifying interactions of rays within the virtualized 3D space.

14. A computer-implemented method comprising:
ray tracing, within a virtualized three dimensional (3D) space, from a light source or camera to a point corresponding to an object that is situated within the virtualized 3D space with an image backplate that is representative of a stereoscopic panorama of an environment;
simulating a finite aperture with respect to a virtual panoramic stereo camera capturing a two dimensional (2D) projection of a 3D scene that corresponds to the image backplate by determining a projection of a ray from the point through a focal point of the virtual panoramic stereo camera to a lens point within the finite aperture of the virtual panoramic stereo camera;
performing a first inverse mapping to obtain an orientation in camera space that corresponds to the point and the lens point by aligning the lens and the focal point with a stereo panoramic part of the projection;
based on the orientation, performing a second inverse mapping to obtain a pixel on a capture plane of the virtual panoramic stereo camera, the pixel corresponding to the point and the lens point; and
rendering an energy contribution from the point at the pixel of the capture plane.

15. The method of claim 14 , wherein the stereoscopic panorama is captured using an omni-directional stereo cylindrical camera.

16. The method of claim 14 , wherein the projection comprises a cylindrical projection and the orientation comprises a camera angle within an xz-plane of the virtualized 3D space associated with the point and a height in a y coordinate in the camera space associated with the point.

17. The method of claim 14 , wherein the stereoscopic panorama is captured using an omni-directional stereo spherical camera.

18. The method of claim 14 , wherein the projection comprises a spherical projection and the orientation comprises a camera angle within an xz-plane of the virtualized 3D space associated with the point and a tilt angle of the virtual panoramic stereo camera with respect to a y-axis the virtualized 3D space associated with the point.

19. The method of claim 14 , wherein the ray tracing comprises determining a path between the light source or camera to the point based on identifying interactions of rays within the virtualized 3D space.

20. The method of claim 14 , wherein the rendering of the energy contribution is part of rendering an image of the 2D projection of the 3D scene.

Assignees

• Nvidia Corp

Inventors

• van Antwerpen Dietger

Classifications

• H04N13/275 Primary

  from three-dimensional [3D] object models, e.g. computer-generated stereoscopic image signals · CPC title

• G06T15/06

  Ray-tracing · CPC title

• G06T15/80 Primary

  Shading · CPC title