Machine-learned models for generating code snippets with predicted placeholders for optimizing software development

Patent metadata

Field	Value
Publication number	US-12333277-B2
Application number	US-202418618371-A
Country	US
Kind code	B2
Filing date	Mar 27, 2024
Priority date	Jun 3, 2022
Publication date	Jun 17, 2025
Grant date	Jun 17, 2025

Abstract

Systems and methods of the present disclosure are directed to a method for machine-learned code segment prediction for optimizing software development. The method includes obtaining an incomplete segment of code. The method includes processing the incomplete segment of code with a machine-learned code prediction model to obtain a sampled set of segment completion predictions that include code that completes the incomplete segment of code. The method includes determining an aggregated segment completion prediction from the sampled set of segment completion predictions. The method includes replacing a portion of the aggregated segment completion prediction with an input field, wherein the portion of the aggregated segment completion prediction is associated with a degree of certainty less than a threshold degree of certainty.

First claim

What is claimed is:

1. A computer-implemented method for machine-learned code segment prediction for optimizing software development, comprising:
obtaining, by a computing system comprising one or more computing devices, an incomplete segment of code;
processing, by the computing system, the incomplete segment of code with a machine-learned code prediction model to obtain a segment completion prediction, wherein the segment completion prediction comprises code that completes the incomplete segment of code and an input field, wherein the input field replaces a portion of the segment completion prediction associated with a degree of certainty less than a threshold degree of certainty; and
providing, by the computing system, the segment completion prediction for display to a user.

2. The computer-implemented method of claim 1 , wherein processing the incomplete segment of code comprises:
processing, by the computing system, the incomplete segment of code with a machine-learned code prediction distillation model to obtain the segment completion prediction, wherein the machine-learned code prediction distillation model is trained based on a teacher model, wherein the teacher model is trained to generate a sampled set of segment completion predictions that are aggregated to obtain an aggregated segment completion prediction, and wherein a portion of the aggregated segment completion prediction associated with the degree of certainty less than the threshold degree of certainty is replaced with an input field.

3. The computer-implemented method of claim 2 , wherein the method further comprises:
evaluating, by the computing system, a distillation loss function that evaluates a difference between the segment completion prediction and a ground truth completion prediction; and
modifying, by the computing system, one or more values of one or more parameters of the machine-learned code prediction distillation model based at least in part on the distillation loss function.

4. The computer-implemented method of claim 3 , wherein the ground truth completion prediction comprises an aggregated segment completion prediction aggregated from a sampled set of completion predictions generated by the teacher model based on the incomplete segment of code, wherein a portion of the aggregated segment completion associated with the degree of certainty less than the threshold degree of certainty replaced with an input field.

5. The computer-implemented method of claim 1 , wherein the input field comprises a suggested portion of code that is selectable by a user.

6. The computer-implemented method of claim 1 , wherein the incomplete segment of code corresponds to a location of a cursor of a user within a development environment.

7. The computer-implemented method of claim 1 , wherein the input field comprises an input field for a machine-learned language model.

8. The computer-implemented method of claim 1 , wherein the machine-learned code prediction model comprises a machine-learned language model, and wherein the segment completion prediction comprises a language output from the machine-learned language model.

9. A computing system for machine-learned code segment prediction for optimizing software development, comprising:
one or more processors;
one or more non-transitory computer-readable media that store instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising:
obtaining an incomplete segment of code;
processing the incomplete segment of code with a machine-learned code prediction model to obtain a segment completion prediction, wherein the segment completion prediction comprises code that completes the incomplete segment of code and an input field, wherein the input field replaces a portion of the segment completion prediction associated with a degree of certainty less than a threshold degree of certainty; and
providing the segment completion prediction.

10. The computing system of claim 9 , wherein processing the incomplete segment of code comprises:
processing the incomplete segment of code with a machine-learned code prediction distillation model to obtain the segment completion prediction, wherein the machine-learned code prediction distillation model is trained based on a teacher model, wherein the teacher model is trained to generate a sampled set of segment completion predictions that are aggregated to obtain an aggregated segment completion prediction, and wherein a portion of the aggregated segment completion prediction associated with the degree of certainty less than the threshold degree of certainty is replaced with an input field.

11. The computing system of claim 10 , wherein the operations further comprise:
evaluating a distillation loss function that evaluates a difference between the segment completion prediction and a ground truth completion prediction; and
modifying one or more values of one or more parameters of the machine-learned code prediction distillation model based at least in part on the distillation loss function.

12. The computing system of claim 11 , wherein the ground truth completion prediction comprises an aggregated segment completion prediction aggregated from a sampled set of completion predictions generated by the teacher model based on the incomplete segment of code, wherein a portion of the aggregated segment completion associated with the degree of certainty less than the threshold degree of certainty replaced with an input field.

13. The computing system of claim 9 , wherein the input field comprises a suggested portion of code that is selectable by a user.

14. The computing system of claim 9 , wherein the incomplete segment of code corresponds to a location of a cursor of a user within a development environment.

15. The computing system of claim 9 , wherein the input field comprises an input field for a machine-learned language model.

16. The computing system of claim 9 , wherein the machine-learned code prediction model comprises a machine-learned language model, and wherein the segment completion prediction comprises a language output from the machine-learned language model.

17. One or more non-transitory computer-readable media that store instructions that, when executed by one or more processors of a computing system, cause the computing system to perform operations, the operations comprising:
obtaining an incomplete segment of code;
processing the incomplete segment of code with a machine-learned code prediction model to obtain a segment completion prediction, wherein the segment completion prediction comprises code that completes the incomplete segment of code and an input field, wherein the input field replaces a portion of the segment completion prediction associated with a degree of certainty less than a threshold degree of certainty; and
providing the segment completion prediction as a suggestion within an Integrated Development Environment (IDE).

18. The one or more non-transitory computer-readable media of claim 17 , wherein processing the incomplete segment of code comprises:
processing the incomplete segment of code with a machine-learned code prediction distillation model to obtain the segment completion prediction, wherein the machine-learned code prediction distillation model is trained based on a teacher model, wherein the teacher model is trained to generate a sampled set of segment completion predictions that are aggregated to obtain an aggregated segment completion prediction, and wherein a portion of the aggregated segment

Assignees

• Google Llc

Inventors

• Johnson Daniel Dun-Ning Woo
• Tarlow Daniel Stefan
• Tabachnyk Maxim
• Rasi Marc Hatcher
• Austin Jacob
• Abolhassani Hassan
• Hegna Jacob Hanson

Classifications

• G06N3/096

  Transfer learning · CPC title

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title

• G06N3/0499

  Feedforward networks · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title