Virtual Reality Parallax Correction

Patent metadata

Field	Value
Publication number	US-2020296348-A1
Application number	US-202016891977-A
Country	US
Kind code	A1
Filing date	Jun 3, 2020
Priority date	Oct 11, 2017
Publication date	Sep 17, 2020
Grant date	—

Abstract

Virtual reality parallax correction techniques and systems are described that are configured to correct parallax for VR digital content captured from a single point of origin. In one example, a parallax correction module is employed to correct artifacts caused in a change from a point of origin that corresponds to the VR digital content to a new viewpoint with respect to an output of the VR digital content. A variety of techniques may be employed by the parallax correction module to correct parallax. Examples of these techniques include depth filtering, boundary identification, smear detection, mesh cutting, confidence estimation, blurring, and error diffusion as further described in the following sections.

First claim

What is claimed is:

1 . In a digital medium environment to correct at least one artifact in virtual reality (VR) digital content, a method implemented by a virtual reality (VR) device, the method comprising:
generating, a perspective detection module implemented by the virtual reality device, perspective data describing a change from an origin perspective with respect to the VR digital content to a changed perspective with respect to the VR digital content; generating, a perspective alteration module implemented by the virtual reality device, a perspective triangle mesh based on the perspective data and an origin triangle mesh associated with the origin perspective; generating, by a parallax correction module implemented by the virtual reality device, a corrected parallax triangle mesh that corrects the at least one artifact in the perspective triangle mesh; and rendering, by a content rendering module implemented by the virtual reality device, the VR digital content based on the corrected parallax triangle mesh.

2 . The method as described in claim 1 , wherein the generating of the corrected parallax triangle mesh by the parallax correction module is based at least in part on depth filtering.

3 . The method as described in claim 1 , wherein the generating of the corrected parallax triangle mesh by the parallax correction module is based at least in part on identification of depth boundaries as part of machine learning through use of a neural network.

4 . The method as described in claim 1 , wherein the generating of the corrected parallax triangle mesh by the parallax correction module is based at least in part on examining triangles included as part of the perspective triangle mesh.

5 . The method as described in claim 4 , wherein the examining is based on a surface normal approximating a view direction of a respective said triangle, a length of an edge of the respective said triangle, or an area of the respective said triangle.

6 . The method as described in claim 1 , wherein the generating of the corrected parallax triangle mesh by the parallax correction module is based at least in part on generating confidence values associated with depths of pixels of the perspective triangle mesh.

7 . The method as described in claim 6 , further comprising blurring portions of VR digital content by the parallax correction module based on respective said confidence values for pixels in respective said portions.

8 . The method as described in claim 6 , wherein the generating of the corrected parallax triangle mesh by the parallax correction module is based at least in part on generating the corrected parallax triangle mesh in which a depth of at least one pixel is described as a distribution of a range of depth values.

9 . In a digital medium environment to correct at least one artifact in virtual reality (VR) digital content, a VR system comprising:
a perspective detection module implemented at least partially in hardware of the VR system to generate perspective data describing a change from an origin perspective to a changed perspective with respect to the VR digital content; a perspective alteration module implemented at least partially in hardware of the VR system to generate a perspective triangle mesh based on the perspective data and an origin triangle mesh associated with the origin perspective; a parallax correction module implemented at least partially in hardware of the VR system to generate a corrected parallax triangle mesh that corrects the at least one artifact in the perspective triangle mesh; and a content rendering module implemented at least partially in hardware of the VR system to render the VR digital content based on the corrected parallax triangle mesh.

10 . The system as described in claim 9 , further comprising a triangle mesh generation module implemented at least partially in hardware of the VR system to generate the origin triangle mesh based on pixels and associated three-dimensional coordinates of the origin perspective with respect to the VR digital content.

11 . The system as described in claim 9 , wherein the parallax correction module further comprises a depth filtering module to generate the corrected parallax triangle mesh based at least in part on depth filtering.

12 . The system as described in claim 9 , wherein the parallax correction module further comprises a boundary identification module to generate the corrected parallax triangle mesh based on identification of depth boundaries as part of machine learning through use of a neural network.

13 . The system as described in claim 9 , wherein the parallax correction module further comprises a smear detection module to generate the corrected parallax triangle mesh based at examination of triangles included as part of the perspective triangle mesh.

14 . The system as described in claim 9 , wherein the parallax correction module further comprises a confidence estimation module to generate confidence values associated with depths of pixels of the perspective triangle mesh.

15 . The system as described in claim 14 , wherein the confidence values are based at least in part on disparity of the pixels in respective stereoscopic images as part of the VR digital content.

16 . The system as described in claim 14 , further comprising a blurring module that is configured to blur portions of VR digital content based on respective said confidence values for pixels in respective said portions.

17 . The system as described in claim 16 , wherein an amount of the blurring of the portions increases for portions that have low confidence values and decreases for portions that have high confidence values.

18 . The system as described in claim 9 , wherein the parallax correction module further comprises an error diffusion module to generate the corrected parallax triangle mesh in which a depth of at least one pixel is described as a distribution of a range of depth values.

19 . The system as described in claim 9 , wherein the parallax correction module further comprises a mesh cutting module to cut the perspective triangle mesh based on identified boundary edges or depth discontinuities.

20 . In a digital medium environment to correct at least one artifact in virtual reality (VR) digital content, a system comprising:
means for generating perspective data describing a change from an origin perspective to a changed perspective with respect to the VR digital content; means for generating a perspective triangle mesh based on the origin triangle mesh and the perspective data; means for generating a corrected parallax triangle mesh that corrects the at least one artifact in the perspective triangle mesh; and means for rendering the VR digital content based on the corrected parallax triangle mesh.

Assignees

• Adobe Inc

Inventors

• Diverdi Stephen Joseph
• Serrano Pacheu Ana Belén
• Hertzmann Aaron Phillip

Classifications

• G06N3/045

  Combinations of networks · CPC title

• H04N2013/0081

  Depth or disparity estimation from stereoscopic image signals · CPC title

• H04N13/366

  using viewer tracking · CPC title

• H04N13/344

  with head-mounted left-right displays · CPC title

• H04N13/332

  Displays for viewing with the aid of special glasses or head-mounted displays [HMD] · CPC title