Visualizing vector graphics in three-dimensional scenes

Patent metadata

Field	Value
Publication number	US-12229892-B2
Application number	US-202318157940-A
Country	US
Kind code	B2
Filing date	Jan 23, 2023
Priority date	Jan 23, 2023
Publication date	Feb 18, 2025
Grant date	Feb 18, 2025

Abstract

In implementations of systems for visualizing vector graphics in three-dimensional scenes, a computing device implements a projection system to receive input data describing a digital image depicting a three-dimensional scene and a vector graphic to be projected into the three-dimensional scene. The projection system generates a depth image by estimating disparity values for pixels of the digital image. A three-dimensional mesh is computed that approximates the three-dimensional scene based on the depth image. The projection system projects the vector graphic onto the digital image by transforming the vector graphic based on the three-dimensional mesh.

First claim

What is claimed is:

1. A method comprising:
receiving, by a processing device, input data describing a digital image depicting a three-dimensional scene and a vector graphic to be projected into the three-dimensional scene;
generating, by the processing device, a depth image by estimating disparity values for pixels of the digital image;
computing, by the processing device, a three-dimensional mesh that approximates the three-dimensional scene based on the depth image; and
projecting, by the processing device, the vector graphic onto the digital image by transforming the vector graphic based on the three-dimensional mesh.

2. The method as described in claim 1 , wherein the three-dimensional mesh is computed based on two-dimensional points of a point cloud.

3. The method as described in claim 2 , wherein the two-dimensional points of the point cloud are sampled from a density distribution generated based on the depth image.

4. The method as described in claim 3 , wherein the density distribution is generated by computing differential density values at the pixels of the digital image.

5. The method as described in claim 1 , further comprising subdividing the three-dimensional mesh using edge-length based subdivision.

6. The method as described in claim 1 , wherein the vector graphic is transformed using a piecewise non-linear transformation.

7. The method as described in claim 1 , further comprising:
computing a vector that is normal to a portion of the three-dimensional scene;
aligning a first axis of the vector graphic with the vector; and
aligning a second axis of the vector graphic with an additional vector that is tangent to the portion of the three-dimensional scene.

8. The method as described in claim 1 , wherein the vector graphic is projected onto the digital image by compositing the vector graphic and the digital image.

9. The method as described in claim 1 , further comprising estimating camera parameters for the digital image by estimating a horizontal field of view and a vertical field of view.

10. The method as described in claim 1 , wherein the three-dimensional mesh is at least one of a triangle mesh, a quadrilateral mesh, or a polygon mesh.

11. A system comprising:
a memory component; and
a processing device coupled to the memory component, the processing device to perform operations comprising:
receiving input data describing a digital image depicting a three-dimensional scene and a vector graphic to be projected into the three-dimensional scene;
generating a point cloud of two-dimensional points based on the three-dimensional scene;
computing a three-dimensional mesh that approximates the three-dimensional scene based on the two-dimensional points; and
projecting the vector graphic onto the digital image by transforming the vector graphic based on the three-dimensional mesh.

12. The system as described in claim 11 , wherein the vector graphic is transformed using a piecewise non-linear transformation.

13. The system as described in claim 11 , wherein the point cloud is generated based on a depth image that includes an estimated disparity value for each pixel of the digital image.

14. The system as described in claim 11 , wherein the vector graphic is projected onto the digital image using a UV mapping and an inverse of the UV mapping.

15. The system as described in claim 11 , wherein the vector graphic is projected onto the digital image by compositing the vector graphic and the digital image.

16. A non-transitory computer-readable storage medium storing executable instructions, which when executed by a processing device, cause the processing device to perform operations comprising:
receiving input data describing a digital image depicting a three-dimensional scene and a vector graphic to be projected into the three-dimensional scene;
generating a depth image by estimating depth values for pixels of the digital image;
computing a three-dimensional mesh that approximates the three-dimensional scene based on the depth image; and
projecting the vector graphic onto the digital image by transforming the vector graphic based on the three-dimensional mesh.

17. The non-transitory computer-readable storage medium as described in claim 16 , wherein the operations further comprise subdividing the three-dimensional mesh using edge-length based subdivision.

18. The non-transitory computer-readable storage medium as described in claim 16 , wherein the operations further comprise estimating camera parameters for the digital image by estimating a horizontal field of view and a vertical field of view.

19. The non-transitory computer-readable storage medium as described in claim 16 , wherein the three-dimensional mesh is computed based on two-dimensional points of a point cloud.

20. The non-transitory computer-readable storage medium as described in claim 16 , wherein the vector graphic is transformed using a piecewise non-linear transformation.

Assignees

• Adobe Inc

Inventors

• Jindal Ashish
• Batra Vineet
• Dhingra Sumit
• Chaudhuri Siddhartha
• Carr Nathan Aaron
• Phogat Ankit

Classifications

• G06T7/80

  Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T2210/56

  Particle system, point based geometry or rendering · CPC title

• G06T15/04

  Texture mapping · CPC title

• G06T17/20 Primary

  Finite element generation, e.g. wire-frame surface description, {tesselation} · CPC title