Animation streaming for media interaction

Patent metadata

Field	Value
Publication number	US-10937220-B2
Application number	US-201916391137-A
Country	US
Kind code	B2
Filing date	Apr 22, 2019
Priority date	Apr 22, 2019
Publication date	Mar 2, 2021
Grant date	Mar 2, 2021

Abstract

Embodiments provide for animation streaming for media interaction by receiving, at a generator, inputs from a target device presenting of a virtual environment; updating, based on the user inputs, a model of the virtual environment; determining network conditions between the generator and target device; generating a packet that includes a forecasted animation set for a virtual object in the updated model that comprises rig updates for the virtual object for at least two different states, and a number of states included in the packet is based on the network conditions; and streaming the packet to the target device, where the target device: receives a second input to interact with the virtual environment that changes the virtual environment to a given state; selects and applies a rig update associated with the given state a local model of the virtual object; and outputs the updated local model on the target device.

First claim

What is claimed is:

1. A method, comprising:
receiving, at a generator, user inputs from a target device displaying a presentation of a virtual environment;
updating, at the generator based on the user inputs, an environmental model of the virtual environment;
determining network conditions between the generator and the target device;
generating a packet that includes a forecasted animation set for a virtual object in the updated environmental model, wherein the forecasted animation set comprises a plurality of rig updates for the virtual object for at least two different states of the environmental model, and a number of states comprising the at least two states is based on the network conditions, wherein the number of states comprising the at least two different states is based on at least one of an available bandwidth between the target device and the generator and a latency between the target device and the generator; and
streaming the packet to the target device.

2. The method of claim 1 , wherein the plurality of rig updates are bitwise compressed before streaming the packet to the target device.

3. The method of claim 1 , wherein the plurality of rig updates excludes data from a static joint of the virtual object.

4. The method of claim 1 , wherein the plurality of rig updates excludes data from a second virtual object that is located outside of a field of view of the virtual environment.

5. The method of claim 1 , wherein receiving the user inputs further comprises decrypting the user inputs according to a first encryption scheme selected to secure communications from the target device to the generator; and
wherein streaming the packet to the target device further comprises encrypting the packet according to a second encryption scheme selected to secure communications from the generator to the target device.

6. The method of claim 1 , wherein the forecasted animation set includes a first rig update to display in response to a first user input advancing the presentation to a first state and a second rig update to display in response to a second user input advancing the presentation to a second state, wherein the first state and the second state are branching candidate states for display at one time.

7. The method of claim 6 , wherein the forecasted animation set includes a third rig update to display when the presentation advances to a subsequent time from the first state and a fourth rig update to display when the presentation advances to the subsequent time from the second state.

8. A method, comprising:
receiving, at a target device displaying a virtual environment, a packet stream from a remote generating device, wherein each packet in the packet stream includes a forecasted animation set for a virtual object included in the virtual environment, wherein the forecasted animation set comprises a first rig update for a first state of the virtual environment at a subsequent time and a second rig update for a second state of the virtual environment at the subsequent time;
receiving, at the target device, a user input to interact with the virtual environment that changes the virtual environment to a given state of the first state and the second state;
selecting a given rig update from the packet stream associated with the given state;
applying the given rig update for the virtual object to a local model of the virtual object displayed by the target device;
outputting, for the subsequent time, the local model on the target device according to the given rig update;
in response to determining that network conditions for the target device indicate one of an increase or a decrease in at least one of:
latency for communication with the remote generating device, and
bandwidth for communication with the remote generating device,

requesting a corresponding increase or decrease in a number of branching states included in each packet; and
transmitting the user input to the remote generating device.

9. The method of claim 8 , wherein the forecasted animation set further comprises:
a third rig update for the first state at a plurality of times subsequent to the subsequent time; and
a fourth rig update for the second state at the plurality of times subsequent to the subsequent time.

10. The method of claim 8 , further comprising:
in response to determining that network conditions for the target device indicate a decrease in latency for communication with the remote generating device, requesting a decrease in a number of branching states included in each packet.

11. The method of claim 8 , further comprising:
in response to determining that network conditions for the target device indicate a decrease in bandwidth for communication with the remote generating device, requesting a decrease in a number of branching states included in each packet.

12. The method of claim 8 , wherein the target device outputs the local model via a first graphics engine and the forecasted animation sets are generated by the generator via a second graphics engine different from the first graphics engine, wherein the target device translates the given rig update from a coordinate system used by the second graphics engine to a coordinate system used by the first graphics engine.

13. The method of claim 8 , further comprising:
decompressing, at the target device the packet stream.

14. The method of claim 8 wherein the user input includes at least one of:
a button push;
a voice command;
a reorientation of the target device in a physical environment to which the virtual environment is overlaid; and
a relocation of the target device in the physical environment.

15. The method of claim 8 , wherein the given rig update excludes positional information for joints of the virtual object that remain static in the virtual environment from a current time to the subsequent time.

16. The method of claim 8 , wherein outputting the local model on the target device according to the given rig update further comprises:
applying a texture to the virtual object from a local cache on the target device.

17. A system, comprising:
a generating device, including a generating processor and a generating memory including generating instructions that when performed by the generating processor enable the generating device to perform an animation operation comprising:
generating a plurality of packets, wherein each packet of the plurality of packets includes a forecasted animation set for a virtual object, wherein the forecasted animation set comprises a plurality of rig updates for the virtual object for at least two different states of a virtual environment for one time;
adjusting a number of rig updates in the forecasted animation set of a given packet based on at least one of a latency and an available bandwidth of a network over which the plurality of packets are streamed; and
streaming the plurality of packets via the network;

a target device, in communication with the generating device via the network, the target device including a target processor and a target memory including target instructions that when performed by the target processor enable the target device to perform an operation comprising:
displaying a local model of the virtual environment that includes the virtual object;
receiving the plurality of packets streamed from the generating device;
receiving a user input to interact with the virtual environment;
transmitting the user input to the generating device;
selecting, based on the user input, a given rig update from the plurality of packets received from the generating devi

Assignees

• Disney Entpr Inc

Inventors

• Mitchell Kenneth J

Classifications

• A63F13/355

  Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client · CPC title

• G06T13/40 Primary

  of characters, e.g. humans, animals or virtual beings · CPC title

• A63F13/358

  Adapting the game course according to the network or server load, e.g. for reducing latency due to different connection speeds between clients · CPC title

• A63F2300/534

  for network load management, e.g. bandwidth optimization, latency reduction · CPC title

• A63F13/573

  using trajectories of game objects, e.g. of a golf ball according to the point of impact · CPC title