Adaptive temporal sampling

Patent metadata

Field	Value
Publication number	US-9734615-B1
Application number	US-201313831092-A
Country	US
Kind code	B1
Filing date	Mar 14, 2013
Priority date	Mar 14, 2013
Publication date	Aug 15, 2017
Grant date	Aug 15, 2017

Abstract

An animation analyzer is configured to receive an animation sequence and to identify a subsample of the frames that are to be rendered. A rendering engine is configured to render the subsample of the frames. The rendering engine is further configured to identify the frames that have not been rendered and to generate in-betweens for the frames that have not been rendered. The rendering engine is further configured to assemble the subsample of frames and the in-betweens into a video sequence depicting the animation sequence.

First claim

What is claimed is:

1. A computer system comprising:
an animation analyzer configured to:
receive an animation sequence for a 3D virtual scene, the animation sequence indicating movements of one or more objects in the 3D scene, the animation sequence including a plurality of frames and each of the plurality of the frames representing the 3D virtual scene at a specific time point from a particular viewpoint;
for each of the plurality frames:
identify an absolute value of a motion vector describing the movement of a pixel from the frame to a next frame in the animation sequence; and
assign a sample rate to the frame based on the identified absolute value such that the plurality of the frames have the same sample rate;

select one or more frames in the plurality of the frames as a first set of key frames in the animation sequence based on the sample rate;
send, to a rendering engine, indication information indicating the first set of key frames in the animation sequence; and

the rendering engine configured to:
receive the animation sequence;
receive, from the animation analyzer, the indication information;
render the first set of key frames; and
generate in-between frames between the first set of key frames based on the rendered first set of key frames; and
assemble the rendered first set of key frames and the generated in-between frames into a video sequence depicting the animation sequence.

2. The computer system of claim 1 , wherein the render engine is further configured to light the first set of key frames and to light the in-between frames based on the lighted first set of key frames.

3. The computer system of claim 1 , wherein selecting the one or more frames in the plurality of the frames as the first set of key frames based on the sample rate comprises sampling the plurality of the frames at the sample rate.

4. The computer system of claim 1 , wherein the render engine is further configured to receive, for each of the frames of the animation sequence, a vector map describing the movement of the pixels from a current frame of the animation sequence to a next frame of the animation sequence; and
wherein, to generate the in-between frames based on the rendered first set of key frames, the render engine is configured to use the vector map to warp pixels in the first set of key frames.

5. The computer system of claim 1 , wherein the render engine is further configured to receive occlusion data for each of rendered frames, the occlusion data stored in two masks for each rendered frame; and
wherein, the render engine is further configured to use the occlusion data to select for source pixels of a rendered frame to warp.

6. The computer system of claim 5 , wherein:
a pixel in a given in-between frame is generated using a previous and a following rendered frames if the pixel is not occluded in either the previous or following rendered frames;
a pixel in a given in-between frame is generated using only the previous rendered frame if the pixel is occluded in the following rendered frame and not occluded in the previous rendered frame; and
a pixel in a given in-between frame is generated using only the following rendered frame if the pixel is occluded in the previous rendered frame and not occluded in the following rendered frame.

7. A computer implemented method comprising:
receiving an animation sequence for a 3D virtual scene, the animation sequence indicating movements of one or more objects in the 3D scene, the animation sequence including a plurality of frames and each of the plurality of the frames representing the 3D virtual scene at a specific time point from a particular viewpoint;
for each of the plurality frames:
identifying an absolute value of a motion vector describing the movement of a pixel from the frame to a next frame in the animation sequence; and
assigning a sample rate to the frame based on the identified absolute value such that the plurality of the frames have the same sample rate;

selecting one or more frames in the plurality of the frames as a first set of key frames based on the sample rate;
sending, to a render engine, indication information indicating the first set of key frames for generating a video sequence depicting the animation sequence based on the indication information, wherein the generation of the video sequence comprises:
rendering the first set of key frames;
generating in-between frames between the first set of key frames based on the rendered first set of key frames; and
assembling the rendered and lit frames into a video sequence depicting the animation sequence.

8. The method of claim 7 , wherein the render engine is further configured to light the first set of key frames and to light the in-between frames based on the lighted first set of key frames.

9. The method of claim 7 , wherein selecting the one or more frames in the plurality of the frames as the first set of key frames based on the sample rate comprises sampling the plurality of the frames at the sample rate.

10. A computer implemented method comprising:
receiving an animation sequence for a 3D virtual scene, the animation sequence indicating movements of one or more objects in the 3D scene, the animation sequence including a plurality of frames and each of the plurality of the frames representing the 3D virtual scene at a specific time point from a particular viewpoint;
for each of the plurality frames:
identifying an absolute value of a motion vector describing the movement of a pixel from the frame to a next frame in the animation sequence; and
assigning a sample rate to the frame based on the identified absolute value such that the plurality of the frames have the same sample rate;

selecting one or more frames in the plurality of the frames as a first set of key frames based on the sample rate;
receiving, from an animation analyzer, indication information indicating the first set of key frames in the animation sequence; and
generating a video sequence depicting the animation sequence based on the indication information, wherein the generation of the video sequence comprises:
rendering, by a render engine, the first set of key frames;
generating the in-between frames between the first set of key frames based on the rendered first set of key frames; and
assembling the rendered first set of key frames and the generated in-between frames into the video sequence depicting the animation sequence.

11. The method of claim 10 further comprising:
receiving, for each of the frames of the animation sequence, a vector map describing the movement of the pixels from a current frame of the animation sequence to a next frame of the animation sequence; and
wherein generating the in-between frames based on the rendered first set of key frames, the render engine is configured to use the vector map to warp pixels in the first set of key frames.

12. The method of claim 11 , wherein:
a pixel in a given in-between frame is generated using a previous and a following rendered frames if the pixel is not occluded in either the previous or following rendered frames;
a pixel in a given in-between frame is generated using only the previous rendered frame if the pixel is occluded in the following rendered frame and not occluded in the previous rendered frame; and
a pixel in a given in-between frame is generated using only the following rendered frame if the pixel is occluded in the previous rendered frame and not occluded in the following rendered frame.

13. A computer system comprising:
an animation analyzer configured to:
receive an animation sequence for a 3D virtual scene, the animation sequence indicating movements

Assignees

• Lucasfilm Entertainment Co Ltd

Inventors

• Knoll John
• Schutz Iv Victor
• Nettleton Mark

Classifications

• G06T13/20 Primary

  Three-dimensional [3D] animation · CPC title

• G06T9/004

  Predictors, e.g. intraframe, interframe coding · CPC title

• G06T19/00

  Manipulating three-dimensional [3D] models or images for computer graphics · CPC title

• G06T13/00

  Animation · CPC title

• G06T13/80 Primary

  Two-dimensional [2D] animation, e.g. using sprites · CPC title