Method, electronic device, and computer program product for processing data

Patent metadata

Field	Value
Publication number	US-12101490-B2
Application number	US-202217680986-A
Country	US
Kind code	B2
Filing date	Feb 25, 2022
Priority date	Jan 21, 2022
Publication date	Sep 24, 2024
Grant date	Sep 24, 2024

Abstract

Embodiments of the present disclosure relate to a method, an electronic device, and a computer program product for processing data. The method includes determining encoded data for a target object by performing hybrid encoding on multiple types of data for the target object, the multiple types of data including image data of the target object. The method further includes determining, by transforming the encoded data, an invariant portion for a shape of the target object and a variable portion for an expression of the target object. The method improves data processing efficiency, saves time and computing resources, and reduces the amount of data transmission.

First claim

What is claimed is:

1. A method for processing data, comprising:
determining encoded data for a target object by performing hybrid encoding on multiple types of data for the target object, the multiple types of data comprising image data of the target object; and
determining, by transforming the encoded data, an invariant portion for a shape of the target object and a variable portion for an expression of the target object;
wherein determining the invariant portion and the variable portion comprises:
performing a first matrix transformation on the encoded data to determine the invariant portion; and
performing a second matrix transformation on the encoded data to determine the variable portion, the second matrix transformation being different from the first matrix transformation.

2. The method according to claim 1 , wherein determining the encoded data comprises:
encoding each type of data in the multiple types of data; and
performing hybrid encoding on the encoded multiple types of data to determine the encoded data.

3. The method according to claim 1 , wherein determining the encoded data comprises:
applying multiple types of data to a cross-modal encoder to perform hybrid encoding on the multiple types of data, the cross-modal encoder being obtained by training with multiple types of sample data as an input and identified sample results as an output.

4. The method according to claim 1 , wherein the multiple types of data further comprise at least one of: voice data or text data.

5. The method according to claim 1 , wherein the variable portion is applicable to another object.

6. The method according to claim 1 , further comprising processing the invariant portion and the variable portion to generate an avatar based on the target object.

7. The method according to claim 1 , wherein the invariant portion and the variable portion are determined in a cloud data center and at least the variable portion is sent from the cloud data center to an avatar rendering module in an edge device.

8. The method according to claim 1 , wherein the variable portion is combined with at least portions of an existing avatar in an avatar pool of an edge device to generate an avatar having the expression of the target object.

9. The method according to claim 1 , wherein the invariant portion and the variable portion are determined at least in part utilizing singular value decomposition (SVD) as applied to a matrix output of a hybrid encoder.

10. The method according to claim 1 , wherein determining the encoded data comprises determining the encoded data in a hybrid encoder having a multiple-stage structure, a first one of the stages of the multiple-stage structure comprising multiple separate encoders configured to encode respective different types of data, and a second one of the stages of the multiple-stage structure comprising a cross-modal encoder configured to perform uniform encoding on the multiple types of data as encoded by respective separate encoders of the first stage of the multiple-stage structure.

11. An electronic device, comprising:
at least one processor; and
a memory coupled to the at least one processor and having instructions stored therein, wherein the instructions, when executed by the at least one processor, cause the electronic device to perform actions comprising:
determining encoded data for a target object by performing hybrid encoding on multiple types of data for the target object, the multiple types of data comprising image data of the target object; and
determining, by transforming the encoded data, an invariant portion for a shape of the target object and a variable portion for an expression of the target object;
wherein determining the invariant portion and the variable portion comprises:
performing a first matrix transformation on the encoded data to determine the invariant portion; and
performing a second matrix transformation on the encoded data to determine the variable portion, the second matrix transformation being different from the first matrix transformation.

12. The electronic device according to claim 11 , wherein determining the encoded data comprises:
encoding each type of data in the multiple types of data; and
performing hybrid encoding on the encoded multiple types of data to determine the encoded data.

13. The electronic device according to claim 11 , wherein determining the encoded data comprises:
applying multiple types of data to a cross-modal encoder to perform hybrid encoding on the multiple types of data, the cross-modal encoder being obtained by training with multiple types of sample data as an input and identified sample results as an output.

14. The electronic device according to claim 11 , wherein the multiple types of data further comprise at least one of: voice data or text data.

15. The electronic device according to claim 11 , wherein the variable portion is applicable to another object.

16. A computer program product tangibly stored on a non-transitory computer-readable medium and including machine-executable instructions, wherein the machine-executable instructions, when executed by a machine, cause the machine to perform steps of a method for processing data, the method comprising:
determining encoded data for a target object by performing hybrid encoding on multiple types of data for the target object, the multiple types of data comprising image data of the target object; and
determining, by transforming the encoded data, an invariant portion for a shape of the target object and a variable portion for an expression of the target object;
wherein determining the invariant portion and the variable portion comprises:
performing a first matrix transformation on the encoded data to determine the invariant portion; and
performing a second matrix transformation on the encoded data to determine the variable portion, the second matrix transformation being different from the first matrix transformation.

17. The computer program product according to claim 16 , wherein determining the encoded data comprises:
encoding each type of data in the multiple types of data; and
performing hybrid encoding on the encoded multiple types of data to determine the encoded data.

18. The computer program product according to claim 16 , wherein determining the encoded data comprises:
applying multiple types of data to a cross-modal encoder to perform hybrid encoding on the multiple types of data, the cross-modal encoder being obtained by training with multiple types of sample data as an input and identified sample results as an output.

19. The computer program product according to claim 16 , wherein the multiple types of data further comprise at least one of: voice data or text data.

20. The computer program product according to claim 16 , wherein the variable portion is applicable to another object.

Assignees

• Dell Products Lp

Inventors

• Wang Zijia
• Sha Danqing
• Jia Zhen

Classifications

• G06T2207/30201

  Face · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T2207/10004

  Still image; Photographic image · CPC title

• H04N19/17 Primary

  the unit being an image region, e.g. an object · CPC title

• H04N19/27 Primary

  involving both synthetic and natural picture components, e.g. synthetic natural hybrid coding [SNHC] · CPC title