Method and apparatus, electronic device, computer-readable storage medium, and computer program product for performing real-time image rendering of virtual objects

Patent metadata

Field	Value
Publication number	US-12437464-B2
Application number	US-202318369721-A
Country	US
Kind code	B2
Filing date	Sep 18, 2023
Priority date	Mar 3, 2022
Publication date	Oct 7, 2025
Grant date	Oct 7, 2025

Abstract

This application provides an image rendering method performed by an electronic device. The method includes: acquiring first texture data of a virtual object and conversion parameters corresponding to second texture data of the virtual object; performing, based on the conversion parameters and the first texture data, fitting rendering to obtain a fitted rendering image comprising the virtual object; updating the conversion parameters and the first texture data based on a rendering loss between the fitted rendering image and a reference rendering image comprising the virtual object , wherein the reference rendering image is a rendering image that is obtained by performing rendering based on the second texture data; and performing, based on the updated conversion parameters and the updated first texture data, real-time rendering to obtain a target rendering image comprising the virtual object.

First claim

What is claimed is:

1. An image rendering method performed by an electronic device and the method comprising:
acquiring, by the electronic device, first texture data of a virtual object and conversion parameters corresponding to second texture data of the virtual object;
performing, by the electronic device, based on the conversion parameters and the first texture data, fitting rendering to obtain a fitted rendering image comprising the virtual object;
updating, by the electronic device, the conversion parameters and the first texture data based on a rendering loss between the fitted rendering image and a reference rendering image comprising the virtual object, wherein the reference rendering image is a rendering image that is obtained by performing rendering based on the second texture data; and
performing, by the electronic device, based on the updated conversion parameters and the updated first texture data, real-time rendering to obtain a target rendering image comprising the virtual object.

2. The method according to claim 1 , wherein the performing, by the electronic device, based on the conversion parameters and the first texture data, fitting rendering to obtain a fitted rendering image comprising the virtual object comprises:
performing, by the electronic device, based on the conversion parameters, space conversion on the first texture data of the virtual object toward the second texture data to obtain third texture data, wherein the third texture data is located in a same dynamic range space as the second texture data; and
performing, by the electronic device, based on the third texture data, fitting rendering to obtain the fitted rendering image comprising the virtual object.

3. The method according to claim 2 , wherein the third texture data comprises a third color value of each texture pixel; and
the performing, by the electronic device, based on the third texture data, fitting rendering corresponding to the virtual object to obtain the fitted rendering image comprising the virtual object comprises:
acquiring, by the electronic device, two-dimensional texture coordinates of the virtual object;
acquiring, by the electronic device, a differentiable rendering framework corresponding to fitting rendering; and
performing, by the electronic device, forward propagation on the two-dimensional texture coordinates and the third color value of each texture pixel in the differentiable rendering framework to obtain the fitted rendering image comprising the virtual object.

4. The method according to claim 1 , wherein the rendering loss between the fitted rendering image and the reference rendering image is defined by:
determining, by the electronic device, an overall pixel value difference between a standard rendering image and the fitted rendering image in a screen space; and
determining, by the electronic device, the rendering loss based on the overall pixel value difference, a length of the fitted rendering image, and a width of the fitted rendering image.

5. The method according to claim 1 , wherein the updating, by the electronic device, the conversion parameters and the first texture data based on the rendering loss comprises:
performing, by the electronic device, based on the rendering loss, partial derivative processing on the first texture data to obtain a gradient corresponding to the first texture data;
performing, by the electronic device, based on the rendering loss, partial derivative processing on the conversion parameters to obtain a gradient corresponding to the conversion parameters; and
updating, by the electronic device, the first texture data based on the gradient corresponding to the first texture data, and updating the conversion parameters based on the gradient corresponding to the conversion parameters.

6. The method according to claim 1 , wherein the performing, by the electronic device, based on the updated conversion parameters and the updated first texture data, real-time rendering on the virtual object comprises:
performing, by the electronic device, when the rendering loss is less than a loss threshold, real-time rendering on the virtual object based on the updated conversion parameters and the updated first texture data; and
performing, by the electronic device, when a number of updates of the conversion parameters and the first texture data reaches a threshold of the number of updates, real-time rendering on the virtual object based on the updated conversion parameters and the updated first texture data.

7. The method according to claim 1 , wherein the performing, by the electronic device, based on the updated conversion parameters and the updated first texture data, real-time rendering on the virtual object to obtain a target rendering image comprising the virtual object comprises:
performing, by the electronic device, based on the updated conversion parameters, space conversion on the updated first texture data toward the second texture data to obtain fourth texture data, wherein the fourth texture data is located in the same dynamic range space as the second texture data;
determining, by the electronic device, at least one two-dimensional texture coordinate of the virtual object;
sampling, by the electronic device, from the fourth texture data, a texture image corresponding to the two-dimensional texture coordinate, and mapping the texture image obtained by sampling; and
generating, by the electronic device, based on a mapping result of each two-dimensional texture coordinate, the target rendering image comprising the virtual object.

8. The method according to claim 1 , wherein a data size of the first texture data is smaller than a data size of the second texture data.

9. The method according to claim 1 , wherein an image information range of the first texture data being smaller than an image information range of the second texture data.

10. An electronic device, comprising:
a memory, configured to store computer-executable instructions; and
a processor, configured to, when executing the computer-executable instructions stored in the memory, cause the electronic device to implement an image rendering method including:
acquiring, by the electronic device, first texture data of a virtual object and conversion parameters corresponding to second texture data of the virtual object;
performing, by the electronic device, based on the conversion parameters and the first texture data, fitting rendering to obtain a fitted rendering image comprising the virtual object;
updating, by the electronic device, the conversion parameters and the first texture data based on a rendering loss between the fitted rendering image and a reference rendering image comprising the virtual object, wherein the reference rendering image is a rendering image that is obtained by performing rendering based on the second texture data; and
performing, by the electronic device, based on the updated conversion parameters and the updated first texture data, real-time rendering to obtain a target rendering image comprising the virtual object.

11. The electronic device according to claim 10 , wherein the performing, by the electronic device, based on the conversion parameters and the first texture data, fitting rendering to obtain a fitted rendering image comprising the virtual object comprises:
performing, by the electronic device, based on the conversion parameters, space conversion on the first texture data of the virtual object toward the second texture data to obtain third texture data, wherein the third texture data is located in a same dynamic range space as the second texture data; and
performing, by the electronic device, based on the third texture data, fitting r

Assignees

• Tencent Tech Shenzhen Co Ltd

Inventors

• Xia Fei
• Ling Fei
• Zhang Yongxiang
• Deng Jun

Classifications

• G06V10/751

  Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching · CPC title

• G06V10/54 Primary

  relating to texture · CPC title

• G06T7/90

  Determination of colour characteristics · CPC title

• G06T9/00

  Image coding (bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00) · CPC title

• G06T2219/2012

  Colour editing, changing, or manipulating; Use of colour codes · CPC title