Pretraining utilizing software dependencies

Patent metadata

Field	Value
Publication number	US-11262985-B2
Application number	US-202016813778-A
Country	US
Kind code	B2
Filing date	Mar 10, 2020
Priority date	Mar 10, 2020
Publication date	Mar 1, 2022
Grant date	Mar 1, 2022

Abstract

In an approach to creating code snippet auto-commenting models utilizing a pre-training model leveraging dependency data, one or more computer processors create a generalized pre-training model trained with one or more dependencies and one or more associated dependency embeddings, wherein dependencies include frameworks, imported libraries, header files, and application programming interfaces associated with a software project. The one or more computer processors create a subsequent model with a model architecture identical to the created pre-training model. The one or more computer processors computationally reduce a training of the created subsequent model utilizing one or more trained parameters, activations, memory cells, and context vectors contained in the created pre-training model. The one or more computer processors create deploy the subsequent model to one to more production environments.

First claim

What is claimed is:

1. A computer-implemented method comprising:
statically analyzing, by one or more computer processors, one or more dependencies associated with software project, wherein the dependencies include frameworks, imported libraries, header files, and application programming interfaces associated with the software project;
creating, by one or more computer processors, one or more dependency embeddings mapping the one or more statically analyzed dependencies;
creating, by one or more computer processors, one or more training sets utilizing the one or more created dependency embeddings;
creating, by one or more computer processors, a generalized pre-training model trained with the one or more training sets, the one or more statically analyzed dependencies and the one or more created dependency embeddings;
creating, by one or more computer processors, a subsequent model with a model architecture identical to the created pre-training model;
training the created subsequent model utilizing the created pre-training model and computationally reducing, by one or more computer processors, the training of the created subsequent model utilizing one or more trained parameters, activations, memory cells, and context vectors associated with the software project contained in the created pre-training model; and
deploying, by one or more computer processors, the subsequent model to one or more production environments.

2. The method of claim 1 , comprising: extracting, by one or more computer processors, one or more imported dependencies associated with the software project; constructing, by one or more computer processors, one or more naming convention dependent tokens; extracting, by one or more computer processors, one or more contexts associated with the software project; and creating, by one or more computer processors, one or more code and comment pairs utilizing the one or more extracted imported dependencies, the one or more constructed naming convention dependent tokens, and the extracted one or more contexts.

3. The method of claim 1 , further comprising:
retraining, by one or more computer processors, one or more downstream software projects utilizing the created pre-training model.

4. The method of claim 1 , wherein the created pre-training model is a recurrent neural network.

5. The method of claim 1 , wherein the subsequent model is an auto-commenting model.

6. The method of claim 5 , wherein the auto-commenting model is a recurrent neural network.

7. The method of claim 6 , further comprises:
integrating, by one or more computer processors, the auto-commenting model into one or more integrated development environments.

8. The method of claim 7 , further comprising:
concurrently creating, by one or more computer processors, one or more code comments utilizing the integrated auto-commenting model; and
displaying, by one or more computer processors, the one or more created code comments.

9. A computer program product comprising: one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the stored program instructions comprising:
program instructions to statically analyze one or more dependencies associated with a software project, wherein the dependencies include frameworks, imported libraries, header files, and application programming interfaces associated with the software project;
program instructions to create one or more dependency embeddings mapping the one or more statically analyzed dependencies;
program instructions to create one or more training sets utilizing the one or more created dependency embeddings;
program instructions to create a generalized pre-training model trained with the one or more training sets, the one or more statically analyzed dependencies and the one or more created dependency embeddings;
program instructions to create a subsequent model with a model architecture identical to the created pre-training model;
program instructions to train the created subsequent model utilizing the created pre-training model and computationally reduce the training of the created subsequent model utilizing one or more trained parameters, activations, memory cells, and context vectors associated with the software projects contained in the created pre-training model; and
program instructions to create deploy the subsequent model to one or more production environments.

10. The computer program product of claim 9 , wherein the subsequent model is an auto-commenting model.

11. The computer program product of claim 10 , wherein the auto-commenting model is a recurrent neural network.

12. The computer program product of claim 10 , wherein the program instructions, stored on the one or more computer readable storage media, comprise:
program instructions to integrate the auto-commenting model into one or more integrated development environments.

13. The computer program product of claim 12 , wherein the program instructions, stored on the one or more computer readable storage media, comprise:
program instructions to concurrently create one or more code comments utilizing the integrated auto-commenting model; and
program instructions to display the one or more created code comments.

14. A computer system comprising: one or more computer processors; one or more computer readable storage media; and program instructions stored on the computer readable storage media for execution by at least one of the one or more processors, the stored program instructions comprising:
program instructions to statically analyze one or more dependencies associated with a software project, wherein the dependencies include frameworks, imported libraries, header files, and application programming interfaces associated with the software project;
program instructions to create one or more dependency embeddings mapping the one or more statically analyzed dependencies;
program instructions to create one or more training sets utilizing the one or more created dependency embeddings;
program instructions to create a generalized pre-training model trained with the one or more training sets, the one or more statically analyzed dependencies and the one or more created dependency embeddings;
program instructions to create a subsequent model with a model architecture identical to the created pre-training model;
program instructions to train the created subsequent model utilizing the created pre-training model and computationally reduce the training of the created subsequent model utilizing one or more trained parameters, activations, memory cells, and context vectors associated with the software projects contained in the created pre-training model; and
program instructions to create deploy the subsequent model to one or more production environments.

15. The computer system of claim 14 , wherein the subsequent model is an auto-commenting model.

16. The computer system of claim 15 , wherein the auto-commenting model is a recurrent neural network.

17. The computer system of claim 15 , wherein the program instructions, stored on the one or more computer readable storage media, comprise:
program instructions to integrate the auto-commenting model into one or more integrated development environments.

Assignees

• IBM

Inventors

• Luo Yan
• Shao Liujia
• Xu Yan
• Fan Sibin

Classifications

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N3/096

  Transfer learning · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06F8/73

  Program documentation · CPC title