Temporal approximation of trilinear filtering

Patent metadata

Field	Value
Publication number	US-11640699-B2
Application number	US-202117396036-A
Country	US
Kind code	B2
Filing date	Aug 6, 2021
Priority date	Aug 6, 2021
Publication date	May 2, 2023
Grant date	May 2, 2023

Abstract

In one embodiment, a method includes receiving instructions to render a snapshot of a scene for a video, where the snapshot is to be displayed using a sequence of N frames, computing a mipmap-level determining factor for a texture appearing in the scene based on a scale of the texture on a pixel grid, selecting a mipmap level of the texture for each of the N frames based on the mipmap-level determining factor, where the mipmap levels selected for the N frames are non-uniform and temporally approximate the mipmap-level determining factor, rendering each of the N frames by sampling the mipmap level of the texture selected for that frame, and displaying the rendered N frames sequentially to represent the snapshot of the scene.

First claim

What is claimed is:

1. A method comprising, by a computing device:
receiving instructions to render a snapshot of a scene for a video, wherein the snapshot is to be displayed using a sequence of N frames;
computing, for a texture appearing in the scene, a mipmap-level determining factor based on a scale of the texture on a pixel grid;
selecting, for each of the N frames, a mipmap level of the texture based on the mipmap-level determining factor and a frame-specific factor repeating at every N frames in the video, wherein the mipmap levels selected for the N frames are non-uniform and temporally approximate the mipmap-level determining factor;
rendering each of the N frames by sampling the mipmap level of the texture selected for that frame; and
displaying the rendered N frames sequentially to represent the snapshot of the scene.

2. The method of claim 1 , wherein the frame-specific factor is determined based on a position of the frame within the sequence of N frames.

3. The method of claim 2 , wherein the position of the frame within the sequence of N frames is determined by taking a modulo operation on a global frame sequence number corresponding to the frame with N.

4. The method of claim 1 , wherein the frame-specific factor within the sequence of N frames are distributed around zero within a range between −0.5 to +0.5.

5. The method of claim 1 , where mipmaps of the texture comprise a plurality of mipmap levels, wherein each mipmap level comprises a rendered array of texels that need to be filtered to determine a color value for each pixel in the pixel grid.

6. The method of claim 5 , wherein a rendered array of texels at mipmap level k is a power of two smaller than a rendered array of texels at mipmap level k-1.

7. The method of claim 5 , wherein sampling the mipmap level of the texture comprises performing a bilinear texture filtering at the mipmap level of the texture.

8. The method of claim 7 , wherein the bilinear texture filtering at the mipmap level of the texture comprises sampling four nearest texels in the mipmap level for a pixel center, wherein a color value for the pixel center is determined by weighted average of the four nearest texels according to their distances to the pixel center.

9. The method of claim 1 , wherein the scale of the texture on the pixel grid in each direction is computed as a Manhattan distance.

10. The method of claim 9 , wherein the scale in U direction is computed as ldx/dul+ldy/dul, wherein x is a horizontal axis of the pixel grid, and wherein y is a vertical axis of the pixel grid.

11. The method of claim 10 , wherein the scale in V direction is computed as ldx/dvl+ldy/dvl.

12. The method of claim 11 , wherein the mipmap-level determining factor for the texture appearing in the scene is computed as max(0, log2(max(scale in U direction, scale in V direction))+MipBias, wherein the MipBias is determined based on an amount of desired blur.

13. The method of claim 1 , wherein N, a number of frames used for displaying the snapshot, is pre-determined to avoid any potential flickering.

14. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:
receive instructions to render a snapshot of a scene for a video, wherein the snapshot is to be displayed using a sequence of N frames;
compute, for a texture appearing in the scene, a mipmap-level determining factor based on a scale of the texture on a pixel grid;
select, for each of the N frames, a mipmap level of the texture based on the mipmap-level determining factor and a frame-specific factor repeating at every N frames in the video, wherein the mipmap levels selected for the N frames are non-uniform and temporally approximate the mipmap-level determining factor;
render each of the N frames by sampling the mipmap level of the texture selected for that frame; and
display the rendered N frames sequentially to represent the snapshot of the scene.

15. The media of claim 14 , wherein the frame-specific factor is determined based on a position of the frame within the sequence of N frames.

16. The media of claim 15 , wherein the position of the frame within the sequence of N frames is determined by taking a modulo operation on a global frame sequence number corresponding to the frame with N.

17. The media of claim 14 , wherein the frame-specific factor within the sequence of N frames are distributed around zero within a range between −0.5 to +0.5.

18. A system comprising: one or more processors; and a non-transitory memory coupled to the processors comprising instructions executable by the processors, the processors operable when executing the instructions to:
receive instructions to render a snapshot of a scene for a video, wherein the snapshot is to be displayed using a sequence of N frames;
compute, for a texture appearing in the scene, a mipmap-level determining factor based on a scale of the texture on a pixel grid;
select, for each of the N frames, a mipmap level of the texture based on the mipmap-level determining factor and a frame-specific factor repeating at every N frames in the video, wherein the mipmap levels selected for the N frames are non-uniform and temporally approximate the mipmap-level determining factor;
render each of the N frames by sampling the mipmap level of the texture selected for that frame; and
display the rendered N frames sequentially to represent the snapshot of the scene.

Assignees

• Meta Platforms Tech Llc

Inventors

• Seiler Larry

Classifications

• G06T15/20

  Perspective computation · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06V40/18

  Eye characteristics, e.g. of the iris · CPC title

• G06V20/20 Primary

  in augmented reality scenes · CPC title

• G06V20/46

  Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames · CPC title