Method of constructing network model for deep learning, device, and storage medium

Patent metadata

Field	Value
Publication number	US-12380333-B2
Application number	US-202117519815-A
Country	US
Kind code	B2
Filing date	Nov 5, 2021
Priority date	Nov 10, 2020
Publication date	Aug 5, 2025
Grant date	Aug 5, 2025

Abstract

A method and apparatus of constructing a network model for deep learning, a device, and a storage medium, which relate to artificial intelligence, and in particular to a field of deep learning. The method of constructing the network model for deep learning includes: determining an execution mode for executing codes, based on a mode parameter; executing the codes by using a first component, which is executable in a first execution mode, through a syntax element in the codes, in response to determining that the execution mode is the first execution mode; and executing the codes by using a second component, which is executable in a second execution mode, through the syntax element, in response to determining that the execution mode is the second execution mode; wherein the first component and the second component have the same component interface, and the syntax element corresponds to the component interface.

First claim

What is claimed is:

1. A method of constructing a network model for deep learning, comprising:
determining an execution mode for executing codes used in constructing the network model, based on a mode parameter;
executing a first syntax element in the codes by using a first component, which is available in a first execution mode, in response to determining that the execution mode is the first execution mode; and
executing the first syntax element in the codes by using a second component, which is available in a second execution mode, in response to determining that the execution mode is the second execution mode, without converting the codes;
wherein when the codes include a second syntax element for enabling the second execution mode, the mode parameter is updated to enable the second execution mode in response to executing the second syntax element, and when the codes include a third syntax element for disabling the second execution mode, the mode parameter is updated to enable the first execution mode in response to executing the third syntax element,
wherein the first component and the second component have the same component interface, and the first syntax element corresponds to the component interface, and
wherein the first execution mode comprises a dynamic graph mode, and the second execution mode comprises a static graph mode.

2. The method of claim 1 , wherein the component interface comprises a control flow component interface; and
wherein the method further comprises: when the first syntax element corresponding to the control flow component interface is executed,
in the first execution mode, using a first control flow component in the first component; and
in the second execution mode, using a second control flow component in the second component.

3. The method of claim 2 , wherein the component interface further comprises a network construction component interface; and
wherein the method further comprises: when the first syntax element corresponding to the network construction component interface is executed,
in the first execution mode, using a network construction component through the first control flow component; and
in the second execution mode, using the network construction component through the second control flow component.

4. The method of claim 1 , wherein the component interface comprises a network execution component interface; and
wherein the method further comprises: when the first syntax element corresponding to the network execution component interface is executed,
in the first execution mode, processing input data by using a first network execution component in the first component, so as to obtain output data; and
in the second execution mode, processing input data by using a second network execution component in the second component, so as to obtain output data.

5. The method of claim 4 , further comprising: when the second network execution component in the second component is executed,
determining a first computational graph for the input data based on the input data; and
processing the input data based on the first computational graph, so as to obtain the output data.

6. The method of claim 1 , wherein the component interface comprises a network optimization component interface; and
wherein the method further comprises: when the first syntax element corresponding to the network optimization component interface is executed,
in the first execution mode, updating a network model parameter by using a first network optimization component in the first component; and
in the second execution mode, updating a network model parameter by using a second network optimization component in the second component.

7. The method of claim 6 , further comprising: when the second network optimization component is executed,
constructing a second computational graph for forward propagation so as to determine a loss caused by the network model parameter;
constructing a third computational graph for backward propagation so as to determine a gradient associated with the network model parameter; and
constructing a fourth computational graph for updating the network model parameter so as to determine an updated network model parameter.

8. The method of claim 1 , wherein the component interface comprises a learning rate adjustment component interface; and
wherein the method further comprises: when the first syntax element corresponding to the learning rate adjustment component interface is executed,
in the first execution mode, adjusting a learning rate by using a first learning rate adjustment component in the first component, so as to update the network model parameter; and
in the second execution mode, adjust a learning rate by using a second learning rate adjustment component in the second component, so as to update the network model parameter.

9. The method of claim 8 , further comprising: when the second learning rate adjustment component is executed,
constructing a fifth computational graph comprising a learning rate node, wherein the fifth computational graph does not comprise a node for updating the learning rate;
receiving an updated value of the learning rate as an input of the learning rate node; and
adjusting the learning rate based on the fifth computational graph and the updated value of the learning rate.

10. The method of claim 1 , wherein the executing the first syntax element in the codes by using the first component comprises:
performing a first operation corresponding to the first component based on input data associated with the first syntax element, so as to determine output data.

11. The method of claim 1 , wherein the executing the first syntax element in the codes by using the second component comprises:
determining a computational graph, which comprises a plurality of network nodes corresponding to a plurality of computing operations, by using a second operation corresponding to the second component; and
determining output data by an execution component using the computational graph, based on input data associated with the first syntax element.

12. An electronic device, comprising:
at least one processor; and
a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions, when executed by the at least one processor, cause the at least one processor to implement the method of claim 1 .

13. A non-transitory computer-readable storage medium having computer instructions stored thereon, wherein the computer instructions allow a computer to implement the method of claim 1 .

Assignees

• Beijing Baidu Netcom Sci & Tech Co Ltd

Inventors

• Wang Haifeng
• Hu Xiaoguang
• Liu Hongyu
• Yu Dianhai
• Ma Yanjun
• Wu Tian

Classifications

• G06N3/045 Primary

  Combinations of networks · CPC title

• G06F8/40

  Transformation of program code · CPC title

• G06N3/063

  using electronic means · CPC title

• G06N3/084 Primary

  Backpropagation, e.g. using gradient descent · CPC title

• G06F8/41 Primary

  Compilation · CPC title