Pretraining utilizing software dependencies

Patent metadata

Field	Value
Publication number	US-2021286598-A1
Application number	US-202016813778-A
Country	US
Kind code	A1
Filing date	Mar 10, 2020
Priority date	Mar 10, 2020
Publication date	Sep 16, 2021
Grant date	—

Abstract

In an approach to creating code snippet auto-commenting models utilizing a pre-training model leveraging dependency data, one or more computer processors create a generalized pre-training model trained with one or more dependencies and one or more associated dependency embeddings, wherein dependencies include frameworks, imported libraries, header files, and application programming interfaces associated with a software project. The one or more computer processors create a subsequent model with a model architecture identical to the created pre-training model. The one or more computer processors computationally reduce a training of the created subsequent model utilizing one or more trained parameters, activations, memory cells, and context vectors contained in the created pre-training model. The one or more computer processors create deploy the subsequent model to one to more production environments.

First claim

What is claimed is:

1 . A computer-implemented method comprising:
creating, by one or more computer processors, a generalized pre-training model trained with one or more dependencies and one or more associated dependency embeddings, wherein dependencies include frameworks, imported libraries, header files, and application programming interfaces associated with a software project; creating, by one or more computer processors, a subsequent model with a model architecture identical to the created pre-training model; computationally reducing, by one or more computer processors, a training of the created subsequent model utilizing one or more trained parameters, activations, memory cells, and context vectors contained in the created pre-training model; and deploying, by one or more computer processors, the subsequent model to one to more production environments.

2 . The method of claim 1 , wherein comprises:
statically analyzing, by one or more computer processors, one or more dependencies associated with the software project; creating, by one or more computer processors, one or more embeddings mapping the one or more statically analyzed dependencies; and creating, by one or more computer processors, one or more training sets utilizing the one or more created embeddings.

3 . The method of claim 1 , wherein statically analyzing one or more dependencies associated with the software project, comprises:
extracting, by one or more computer processors, one or more imported dependencies; constructing, by one or more computer processors, one or more naming convention dependent tokens; extracting, by one or more computer processors, one or more contexts; and creating, by one or more computer processors, one or more code and comment pairs utilizing the one or more extracted imported dependencies, the one or more constructed naming convention dependent tokens, and the extracted one or more contexts.

4 . The method of claim 1 , further comprising:
retraining, by one or more computer processors, one or more downstream software projects utilizing the created pre-training model.

5 . The method of claim 1 , wherein the created pre-training model is a recurrent neural network.

6 . The method of claim 1 , wherein the subsequent model is an auto-commenting model.

7 . The method of claim 6 , wherein the auto-commenting model is a recurrent neural network.

8 . The method of claim 7 , further comprises:
integrating, by one or more computer processors, the auto-commenting model into one or more integrated development environments.

9 . The method of claim 8 , further comprising:
concurrently creating, by one or more computer processors, one or more code comments utilizing the integrated auto-commenting model; and displaying, by one or more computer processors, one or more created code comment

10 . A computer program product comprising:
one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the stored program instructions comprising: program instructions to create a generalized pre-training model trained with one or more dependencies and one or more associated dependency embeddings, wherein dependencies include frameworks, imported libraries, header files, and application programming interfaces associated with a software project; program instructions to create a subsequent model with a model architecture identical to the created pre-training model; program instructions to computationally reduce a training of the created subsequent model utilizing one or more trained parameters, activations, memory cells, and context vectors contained in the created pre-training model; and program instructions to create deploy the subsequent model to one to more production environments.

11 . The computer program product of claim 10 , wherein the program instructions, stored on the one or more computer readable storage media, comprise:
program instructions to statically analyze one or more dependencies associated with the software project; program instructions to create one or more embeddings mapping the one or more statically analyzed dependencies; and program instructions to create one or more training sets utilizing the one or more created embeddings.

12 . The computer program product of claim 10 , wherein the subsequent model is an auto-commenting model.

13 . The computer program product of claim 12 , wherein the auto-commenting model is a recurrent neural network.

14 . The computer program product of claim 12 , wherein the program instructions, stored on the one or more computer readable storage media, comprise:
program instructions to integrate the auto-commenting model into one or more integrated development environments.

15 . The computer program product of claim 14 , wherein the program instructions, stored on the one or more computer readable storage media, comprise:
program instructions to concurrently create one or more code comments utilizing the integrated auto-commenting model; and program instructions to display one or more created code comment

16 . A computer system comprising:
one or more computer processors; one or more computer readable storage media; and program instructions stored on the computer readable storage media for execution by at least one of the one or more processors, the stored program instructions comprising:
program instructions to create a generalized pre-training model trained with one or more dependencies and one or more associated dependency embeddings, wherein dependencies include frameworks, imported libraries, header files, and application programming interfaces associated with a software project; and
program instructions to create a subsequent model with a model architecture identical to the created pre-training model;
program instructions to computationally reduce a training of the created subsequent model utilizing one or more trained parameters, activations, memory cells, and context vectors contained in the created pre-training model; and
program instructions to create deploy the subsequent model to one to more production environments.

17 . The computer system of claim 16 , wherein the program instructions, stored on the one or more computer readable storage media, comprise:
program instructions to statically analyze one or more dependencies associated with the software project; program instructions to create one or more embeddings mapping the one or more statically analyzed dependencies; and program instructions to create one or more training sets utilizing the one or more created embeddings.

18 . The computer system of claim 16 , wherein the subsequent model is an auto-commenting model.

19 . The computer system of claim 18 , wherein the auto-commenting model is a recurrent neural network.

20 . The computer system of claim 18 , wherein the program instructions, stored on the one or more computer readable storage media, comprise:
program instructions to integrate the auto-commenting model into one or more integrated development environments.

Assignees

• IBM

Inventors

• Luo Yan
• Shao Liujia
• Xu Yan
• Fan Sibin

Classifications

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N3/096

  Transfer learning · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title