Generation of three-dimensional imagery to supplement existing content

Patent metadata

Field	Value
Publication number	US-9897806-B2
Application number	US-201514618990-A
Country	US
Kind code	B2
Filing date	Feb 10, 2015
Priority date	Feb 10, 2015
Publication date	Feb 20, 2018
Grant date	Feb 20, 2018

Abstract

A computer-implemented process for generating stereoscopic images to supplement existing content includes obtaining first image data and obtaining second image data comprising a plurality of sample points. A direction, a color value, and a depth value are associated with each sample point. The direction and depth values are relative to a common origin. A mesh is generated such that the mesh is displaced from the origin in the directions associated with the sample points by distances representative of the corresponding depth values. The second image data is mapped to the mesh such that the color values associated with the sample points are mapped to the mesh at the corresponding directions. First and second stereoscopic images of the mesh are generated from first and second perspectives, respectively. The first image data is caused to be displayed on a first portion of a display. At least part of both the first stereoscopic image and the second stereoscopic image is caused to be displayed on a second portion of the display. At least part of the second portion of the display does not overlap the first portion of the display.

First claim

What is claimed is:

1. A computer-implemented method for generating stereoscopic images to supplement existing content, the method comprising:
obtaining first image data;
obtaining second image data comprising a plurality of sample points,
wherein a direction, a color value, and a depth value are associated with each sample point, and
wherein the directions and depth values are relative to a common origin;

generating a mesh, wherein generating the mesh includes displacing the sample points from the common origin in the directions associated with the sample points by distances representative of the corresponding depth values;
mapping the second image data to the mesh, wherein the color values associated with the sample points are mapped to the mesh at the corresponding directions;
generating a first stereoscopic image of an inner surface of the mesh from a first perspective;
generating a second stereoscopic image of the inner surface of the mesh from a second perspective;
causing a display of the first image data on a first portion of a display; and
causing a display of at least part of the first stereoscopic image and at least part of the second stereoscopic image on a second portion of the display,
wherein at least part of the second portion of the display surrounds the first portion of the display.

2. The computer-implemented method of claim 1 , wherein the display of the first and second stereoscopic images creates an illusion of depth.

3. The computer-implemented method of claim 1 , wherein the first image data and the first and second images are displayed on a head-mounted display.

4. The computer-implemented method of claim 1 , further comprising tessellating the second image data, wherein tessellating the second image data creates a plurality of vertices, and wherein the vertices are used as the sample points for generating the mesh, mapping the image data to the mesh, and generating the first and second stereoscopic images of the inner surface of the mesh.

5. The computer-implemented method of claim 4 , wherein the density of vertices is greater than a density of pixels of a display to be used to display the first and second stereoscopic images.

6. The computer-implemented method of claim 1 , further comprising:
determining a portion of an image represented by the second image data, wherein the obtained second image data includes only the data for the portion of the image.

7. The computer-implemented method of claim 6 , wherein the portion of the image is determined at least in part by the position of a head-mounted display.

8. The computer-implemented method of claim 1 , wherein the second image data represents an image of a scene from the perspective of a vantage point.

9. The computer-implemented method of claim 8 , wherein the scene is computer-generated.

10. The computer-implemented method of claim 8 , wherein the scene is a live scene.

11. The computer-implemented method of claim 8 , wherein the image of the scene includes a 360 degree view horizontally around the vantage point and a 180 degree view vertically around the vantage point.

12. A non-transitory computer-readable storage medium for generating stereoscopic images, the non-transitory computer-readable storage medium comprising computer-executable instructions for:
obtaining first image data;
obtaining second image data comprising a plurality of sample points,
wherein a direction, a color value, and a depth value are associated with each sample point, and
wherein the directions and depth values are relative to a common origin;

generating a mesh, wherein generating the mesh includes displacing the sample points from the common origin in the directions associated with the sample points by distances representative of the corresponding depth values;
mapping the second image data to the mesh, wherein the color values associated with the sample points are mapped to the mesh at the corresponding directions;
generating a first stereoscopic image of an inner surface of the mesh from a first perspective;
generating a second stereoscopic image of the inner surface of the mesh from a second perspective;
causing a display of the first image data on a first portion of a display; and
causing a display of at least part of the first stereoscopic image and at least part of the second stereoscopic image on a second portion of the display,
wherein at least part of the second portion of the display surrounds the first portion of the display.

13. The non-transitory computer-readable storage medium of claim 12 , wherein the display of the first and second stereoscopic images creates an illusion of depth.

14. The non-transitory computer-readable storage medium of claim 12 , wherein the first image data and the first and second images are displayed on a head-mounted display.

15. The non-transitory computer-readable storage medium of claim 12 , further comprising computer-executable instructions for:
tessellating the second image data,
wherein tessellating the second image data creates a plurality of vertices, and
wherein the vertices are used as the sample points for generating the mesh, mapping the image data to the mesh, and generating the first and second stereoscopic images of the inner surface of the mesh.

16. The non-transitory computer-readable storage medium of claim 15 , wherein the density of vertices is greater than a density of pixels of a display to be used to display the first and second stereoscopic images.

17. The non-transitory computer-readable storage medium of claim 12 , further comprising computer-executable instructions for:
determining a portion of an image represented by the second image data, wherein the obtained second image data includes only the data for the portion of the image.

18. The non-transitory computer-readable storage medium of claim 17 , wherein the portion of the image is determined at least in part by the position of a head-mounted display.

19. The non-transitory computer-readable storage medium of claim 12 , wherein the second image data represents an image of a scene from the perspective of a vantage point.

20. The non-transitory computer-readable storage medium of claim 19 , wherein the scene is computer-generated.

21. The non-transitory computer-readable storage medium of claim 19 , wherein the scene is a live scene.

22. The non-transitory computer-readable storage medium of claim 19 , wherein the image of the scene includes a 360 degree view horizontally around the vantage point and a 180 degree view vertically around the vantage point.

23. A system for generating stereoscopic images, the system comprising:
a display; and
one or more processors coupled to the display and configured to:
obtain first image data;
obtain second image data comprising a plurality of sample points,
wherein a direction, a color value, and a depth value are associated with each sample point, and
wherein the directions and depth values are relative to a common origin;

generate a mesh, wherein generating the mesh includes displacing the sample points from the common origin in the directions associated with the sample points by distances representative of the corresponding depth values;
map the second image data to the mesh, wherein the color values associated with the sample points are mapped to the mesh at the corresponding directions;
generate a first stereoscopic image of an inner surface of the mesh from a first perspective;
generate a

Assignees

• Dreamworks Animation Llc
• Dreamworks Animation L L C

Inventors

• Herman Brad Kenneth

Classifications

• G02B27/017 Primary

  Head mounted · CPC title

• H04N13/279 Primary

  the virtual viewpoint locations being selected by the viewers or determined by tracking · CPC title

• G06F3/01

  Input arrangements or combined input and output arrangements for interaction between user and computer (G06F3/16 takes precedence) · CPC title

• G06T2215/16

  Using real world measurements to influence rendering · CPC title

• H04N13/344

  with head-mounted left-right displays · CPC title