Cloud-client rendering method based on adaptive virtualized rendering pipeline

Patent metadata

Field	Value
Publication number	US-11989797-B2
Application number	US-202117627677-A
Country	US
Kind code	B2
Filing date	Jan 6, 2021
Priority date	Mar 11, 2020
Publication date	May 21, 2024
Grant date	May 21, 2024

Abstract

The present invention discloses a cloud-client rendering computing method based on an adaptive virtualized rendering pipeline, comprising the following steps of: defining a rendering pipeline, including defining a rendering resource, a rendering algorithm, and a read-write relationship between the rendering algorithm and the rendering resource; selecting an optimal cloud-client computing distribution solution in a real-time manner from a cloud-client computing distribution solution set comprising each rendering resource that is allocated to a cloud or client for computing, based on self-defined optimization objectives and an optimization budget of a framework user; and executing a corresponding rendering algorithm on cloud and/or on a client according to the cloud-client computing distribution solution, thereby obtaining a rendering result. The rendering framework can adaptively select the cloud-client computing distribution solution upon cloud-client combined rendering and dynamically adjust it with a change of an optimization parameter.

First claim

What is claimed is:

1. A cloud-client rendering computing method based on an adaptive virtualized rendering pipeline, comprising the following steps of:
defining a rendering pipeline with an explicit representation manner for a resource, comprising defining a rendering resource, a rendering algorithm, and a read-write relationship between the rendering algorithm and the rendering resource, wherein the rendering resource comprises an input scene, a middle computing result, and a final screen resource, and each rendering pipeline comprises at least one set of a plurality of rendering algorithms jointly acting on one same rendering resource;
selecting an optimal cloud-client computing distribution solution in a real-time manner from a cloud-client computing distribution solution set comprising each rendering resource that is allocated to a cloud or client for computing, based on a framework user's six self-defined optimization objectives comprised of quality, energy consumption, performance, delay, bandwidth and space, and budget of the optimization objectives; and
executing a corresponding rendering algorithm on cloud and/or on a client according to the cloud-client computing distribution solution, thereby obtaining a rendering result;
wherein the selecting an optimal cloud-client computing distribution solution in a real-time manner from a cloud-client computing distribution solution set comprising each rendering resource that is allocated to a cloud or client for computing comprises: enumeratively selecting whether to perform on-cloud computing for each rendering resource represented by a rendering pipeline, wherein a distribution of whether all rendering resources are computed on cloud forms a cloud-client computing distribution solution, thereby composing a cloud-client computing distribution solution set; removing an invalid cloud-client computing distribution solution from the cloud-client computing distribution solution set according to a read-write relationship between the rendering algorithm and the rendering resource; selecting all configurations satisfying Pareto Optimality according to the optimization objectives from a remaining cloud-client computing distribution solution, thereby obtaining a series of superior cloud-client computing distribution solutions; and upon operation, selecting one optimal cloud-client computing distribution solution from the series of superior cloud-client distribution solutions according to an operation budget.

2. The cloud-client rendering computing method based on an adaptive virtualized rendering pipeline as claimed in claim 1 , wherein when all rendering resources in the cloud-client computing distribution solution are executed on a client, a corresponding rendering algorithm is executed by the client according to a defined rendering pipeline to obtain a rendering result.

3. The cloud-client rendering computing method based on an adaptive virtualized rendering pipeline as claimed in claim 1 , wherein when a part of all rendering resources in the cloud-client computing distribution solution is executed on cloud and the remaining is executed on a client, a corresponding rendering algorithm is executed on cloud according to a defined rendering pipeline to obtain a computing result, and then the computing result is transmitted to the client, wherein the client executes a corresponding rendering algorithm according to the received computing result to obtain a rendering result.

4. The cloud-client rendering computing method based on an adaptive virtualized rendering pipeline as claimed in claim 1 , wherein when all rendering resources in the cloud-client computing distribution solution are executed on cloud, a corresponding rendering algorithm is executed on cloud according to a defined rendering pipeline to obtain a final display resource, wherein the client directly displays the received final display resource.

5. The cloud-client rendering computing method based on an adaptive virtualized rendering pipeline as claimed in claim 3 , wherein the computing result obtained by the corresponding rendering algorithm executed on cloud is transmitted to the client after being compressed with a compression technique.

6. The cloud-client rendering computing method based on an adaptive virtualized rendering pipeline as claimed in claim 1 , wherein the rendering computing method further comprises: monitoring a change of the optimization objectives in a real-time manner, dynamically updating the optimization objectives according to the change of the optimization objectives and updating a currently optimal cloud-client computing distribution solution according to the updated optimization objectives.

7. The cloud-client rendering computing method based on an adaptive virtualized rendering pipeline as claimed in claim 5 , wherein the rendering computing method further comprises: monitoring a change of the optimization objectives in a real-time manner, dynamically updating the optimization objectives according to the change of the optimization objectives and updating a currently optimal cloud-client computing distribution solution according to the updated optimization objectives.

8. The cloud-client rendering computing method based on an adaptive virtualized rendering pipeline as claimed in claim 6 , wherein a network transmission bandwidth is monitored in a real-time manner, the optimization objectives are dynamically updated when a network transmission delay increases, and a currently optimal cloud-client computing distribution solution is updated according to the updated optimization objectives.

9. The cloud-client rendering computing method based on an adaptive virtualized rendering pipeline as claimed in claim 7 , wherein a network transmission bandwidth is monitored in a real-time manner, the optimization objectives are dynamically updated when a network transmission delay increases, and a currently optimal cloud-client computing distribution solution is updated according to the updated optimization objectives.

10. The cloud-client rendering computing method based on an adaptive virtualized rendering pipeline as claimed in claim 4 , wherein the computing result obtained by the corresponding rendering algorithm executed on cloud is transmitted to the client after being compressed with a compression technique.

Assignees

• Univ Zhejiang

Inventors

• Bao Hujun
• Wang Rui
• LAN Weiju

Classifications

• G06T11/26

  Drawing of charts or graphs · CPC title

• G06T11/10 Primary

  Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) · CPC title

• G06T1/20 Primary

  Processor architectures; Processor configuration, e.g. pipelining · CPC title

• H04L67/10

  in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46) · CPC title