Virtual reality parallax correction

Patent metadata

Field	Value
Publication number	US-11050994-B2
Application number	US-202016891977-A
Country	US
Kind code	B2
Filing date	Jun 3, 2020
Priority date	Oct 11, 2017
Publication date	Jun 29, 2021
Grant date	Jun 29, 2021

Abstract

Virtual reality parallax correction techniques and systems are described that are configured to correct parallax for VR digital content captured from a single point of origin. In one example, a parallax correction module is employed to correct artifacts caused in a change from a point of origin that corresponds to the VR digital content to a new viewpoint with respect to an output of the VR digital content. A variety of techniques may be employed by the parallax correction module to correct parallax. Examples of these techniques include depth filtering, boundary identification, smear detection, mesh cutting, confidence estimation, blurring, and error diffusion as further described in the following sections.

First claim

What is claimed is:

1. In a digital medium environment to correct at least one artifact in virtual reality (VR) digital content, a method implemented by a virtual reality (VR) device, the method comprising:
detecting, by the virtual reality device, a change in perspective caused by movement away from a point of origin;
generating, by the virtual reality device, perspective data based on the detected change, the perspective data describing a change from an origin perspective with respect to the VR digital content to a changed perspective with respect to the VR digital content;
generating, by the virtual reality device, a perspective triangle mesh based on the perspective data and an origin triangle mesh associated with the origin perspective;
generating, by the virtual reality device, a corrected parallax triangle mesh that corrects the at least one artifact in the perspective triangle mesh, the artifact caused by the detected change in perspective and the generating of the corrected parallax triangle mesh is based at least in part on generating confidence values associated with depths of pixels of the perspective triangle mesh; and
rendering, by the virtual reality device, the VR digital content based on the corrected parallax triangle mesh.

2. The method as described in claim 1 , wherein the generating of the corrected parallax triangle mesh is based at least in part on depth filtering.

3. The method as described in claim 1 , wherein the generating of the corrected parallax triangle mesh is based at least in part on identification of depth boundaries as part of machine learning through use of a neural network.

4. The method as described in claim 1 , wherein the generating of the corrected parallax triangle mesh is based at least in part on examining triangles included as part of the perspective triangle mesh.

5. The method as described in claim 4 , wherein the examining is based on a surface normal approximating a view direction of a respective said triangle, a length of an edge of the respective said triangle, or an area of the respective said triangle.

6. The method as described in claim 5 , further comprising blurring portions of VR digital content based on respective said confidence values for pixels in respective said portions.

7. The method as described in claim 5 , wherein the generating of the corrected parallax triangle mesh is based at least in part on generating the corrected parallax triangle mesh in which a depth of at least one pixel is described as a distribution of a range of depth values.

8. In a digital medium environment to correct at least one artifact in virtual reality (VR) digital content, a VR system comprising:
a perspective detection module implemented by a processing system of the VR system to:
detect a change in perspective caused by movement away from a point of origin; and
generate perspective data based on the detected change, the perspective data describing a change from an origin perspective to a changed perspective with respect to the VR digital content;

a perspective alteration module implemented by the processing system of the VR system to generate a perspective triangle mesh based on the perspective data and an origin triangle mesh associated with the origin perspective;
a parallax correction module implemented by the processing system of the VR system to generate a corrected parallax triangle mesh that corrects the at least one artifact in the perspective triangle mesh, the artifact caused by the detected change in perspective, the parallax correction module including a smear detection module configured to generate the corrected parallax triangle mesh based on examination of triangles included as part of the perspective triangle mesh; and
a content rendering module implemented by the processing system of the VR system to render the VR digital content based on the corrected parallax triangle mesh.

9. The system as described in claim 8 , further comprising a triangle mesh generation module implemented by the processing system of the VR system to generate the origin triangle mesh based on pixels and associated three-dimensional coordinates of the origin perspective with respect to the VR digital content.

10. The system as described in claim 8 , wherein the parallax correction module further comprises a depth filtering module to generate the corrected parallax triangle mesh based at least in part on depth filtering.

11. The system as described in claim 8 , wherein the parallax correction module further comprises a boundary identification module to generate the corrected parallax triangle mesh based on identification of depth boundaries as part of machine learning through use of a neural network.

12. The system as described in claim 8 , wherein the smear detection module is configured to generate the corrected parallax triangle mesh by examination of the triangles based on a surface normal approximating a view direction of a respective said triangle, a length of an edge of the respective said triangle, or an area of the respective said triangle.

13. The system as described in claim 8 , wherein the parallax correction module further comprises a confidence estimation module to generate confidence values associated with depths of pixels of the perspective triangle mesh.

14. The system as described in claim 13 , wherein the confidence values are based at least in part on disparity of the pixels in respective stereoscopic images as part of the VR digital content.

15. The system as described in claim 13 , further comprising a blurring module that is configured to blur portions of VR digital content based on respective said confidence values for pixels in respective said portions.

16. The system as described in claim 15 , wherein an amount of the blurring of the portions increases for portions that have low confidence values and decreases for portions that have high confidence values.

17. The system as described in claim 8 , wherein the parallax correction module further comprises an error diffusion module to generate the corrected parallax triangle mesh in which a depth of at least one pixel is described as a distribution of a range of depth values.

18. The system as described in claim 8 , wherein the parallax correction module further comprises a mesh cutting module to cut the perspective triangle mesh based on identified boundary edges or depth discontinuities.

19. A computer-readable non-transitory storage medium comprising instructions stored thereon that, responsive to execution by a computing device, causes the computing device to perform operations including:
detecting a change in perspective caused by movement away from a point of origin with respect to VR digital content;
generating perspective data based on the detected change, the perspective data describing a change from an origin perspective to a changed perspective with respect to the VR digital content;
generating a perspective triangle mesh based on the origin triangle mesh and the perspective data;
generating a corrected parallax triangle mesh that corrects at least one artifact in the perspective triangle mesh based on depth filtering, the artifact caused by the detected change in perspective; and
rendering the VR digital content based on the corrected parallax triangle mesh.

20. The computer-readable non-transitory storage medium as described in claim 19 , the operations further comprising generating the origin triangle mesh based on pixels and associated three-dimensional coordinates of the origin perspective wit

Assignees

• Adobe Inc

Inventors

• Diverdi Stephen Joseph
• Serrano Pacheu Ana Belén
• Hertzmann Aaron Phillip

Classifications

• G06N3/045

  Combinations of networks · CPC title

• H04N2013/0081

  Depth or disparity estimation from stereoscopic image signals · CPC title

• H04N13/366

  using viewer tracking · CPC title

• H04N13/344

  with head-mounted left-right displays · CPC title

• H04N13/332

  Displays for viewing with the aid of special glasses or head-mounted displays [HMD] · CPC title