Recommended methods, devices, electronic devices, and storage media for large model interface configuration

Patent metadata

Field	Value
Publication number	US-2024411552-A1
Application number	US-202418748345-A
Country	US
Kind code	A1
Filing date	Jun 20, 2024
Priority date	May 22, 2024
Publication date	Dec 12, 2024
Grant date	—

Abstract

A computer-implemented method for recommending a large model interface configuration includes: obtaining a search space of a model interface configuration and a test data set, wherein the search space comprises at least one candidate model interface and a value range of a hyperparameter; and obtaining a plurality of model interface configuration sets based on the search space, wherein each model interface configuration set comprises a candidate model interface and a value of the hyperparameter; and obtaining a test result corresponding to each model interface configuration set, by using the test data set to test a large model called based on each model interface configuration set; and determining a target interface configuration based on the test results corresponding to the plurality of model interface configuration sets.

First claim

1 . A computer-implemented method for recommending a large model interface configuration, the method comprising:
obtaining a search space of a model interface configuration and a test data set, wherein the search space comprises at least one candidate model interface and a value range of a hyperparameter; and obtaining a plurality of model interface configuration sets based on the search space, wherein each model interface configuration set comprises a candidate model interface and a value of the hyperparameter; and obtaining a test result corresponding to each model interface configuration set, by using the test data set to test a large model called based on each model interface configuration set; and determining a target interface configuration based on the test results corresponding to the plurality of model interface configuration sets.

2 . The method according to claim 1 , wherein determining the target interface configuration based on the test results corresponding to the plurality of model interface configuration sets comprises:
obtaining a first evaluation identifier and obtaining a first evaluation rule based on the first evaluation identifier, wherein the first evaluation rule comprises an item to be evaluated; and obtaining an evaluation result corresponding to each model interface configuration set, by evaluating the item to be evaluated in the test result corresponding to each model interface configuration set based on the first evaluation rule; and determining the target interface configuration based on the evaluation results corresponding to the plurality of model interface configuration sets.

3 . The method according to claim 2 , wherein determining the target interface configuration based on the evaluation results corresponding to the plurality of model interface configuration sets comprises:
searching for a new interface configuration in the search space based on the evaluation results corresponding to the plurality of model interface configuration sets; obtaining a test result corresponding to the new interface configuration by using the test data set; obtaining an evaluation result corresponding to the new interface configuration, by evaluating the item to be evaluated in the test result corresponding to the new interface configuration based on the first evaluation rule; and continuing to search for another new interface configuration in the search space based on the plurality of model interface configuration sets, the evaluation results corresponding to the plurality of model interface configuration sets, the test result corresponding to the new interface configuration and the evaluation result corresponding to the new interface configuration, until a preset number of interface configurations are searched, and selecting the target interface configuration from the preset number of interface configurations based on evaluation results corresponding to the preset number of interface configurations.

4 . The method according to claim 3 , wherein searching for the new interface configuration in the search space based on the evaluation results corresponding to the plurality of model interface configuration sets comprises:
obtaining a first output requirement for the model interface configuration, wherein the first output requirement comprises at least one of: a goal of recommending the model interface configuration, a requirement for a number of model interface configurations, a first condition that the model interface configuration satisfies, and a format requirement for the model interface configuration; generating first prompt information based on the plurality of model interface configuration sets, the evaluation results corresponding to the plurality of model interface configuration sets, the search space, and the first output requirement; and obtaining the new interface configuration by inputting the first prompt information into the large model for processing.

5 . The method according to claim 3 , wherein the evaluation result comprises an evaluation score for the item to be evaluated, and searching for the new interface configuration in the search space based on the evaluation results corresponding to the plurality of model interface configuration sets comprises:
obtaining a transformed evaluation score by performing data transformation on the evaluation score corresponding to each model interface configuration set with a method for power transformation; and searching for the new interface configuration in the search space by using the large model based on the transformed evaluation scores corresponding to the plurality of model interface configuration sets.

6 . The method according to claim 2 , wherein determining the target interface configuration based on the evaluation results corresponding to the plurality of model interface configuration sets comprises:
determining a goal of recommending the model interface configuration that matches the item to be evaluated; and determining the target interface configuration from the plurality of model interface configuration sets based on the goal of recommending the model interface configuration, and the evaluation results corresponding to the plurality of model interface configuration sets.

7 . The method according to claim 1 , wherein obtaining the plurality of model interface configuration sets based on the search space comprises:
obtaining a second output requirement for the model interface configuration, wherein the second output requirement for the model interface configuration comprises at least one of: a second condition that the model interface configuration satisfies, a requirement for a number of model interface configurations, and a format requirement for the model interface configuration; and generating second prompt information based on the search space and the second output requirement; and obtaining the plurality of model interface configuration sets by inputting the second prompt information into the large model for processing.

8 . The method according to claim 1 , wherein obtaining the test result corresponding to each model interface configuration set, by using the test data set to test the large model called based on each model interface configuration set comprises:
obtaining an interface function corresponding to each model interface configuration set by transforming first interface call information corresponding to the candidate model interface in each model interface configuration set, wherein the first interface call information comprises a value of an input parameter, and a method of parsing an output parameter; and obtaining an interface call output result by calling the interface function corresponding to each model interface configuration set and using the test data set to test the large model called based on each model interface configuration set; and parsing a model output result from the interface call output result based on the method of parsing the output parameter; and obtaining the test result corresponding to each model interface configuration set based on the model output result.

9 . The method according to claim 1 , further comprising:
obtaining second interface call information to be registered, and verifying the second interface call information, wherein the second interface call information comprises an interface identifier; and determining whether the interface identifier exists in a first database in case that the verifying of the second interface call information passes; and storing the second interface call information into the first database in case that the interface identifier does not exist in the first database; and updating interface call information corresponding to the interface identifier

Assignees

• Beijing Baidu Netcom Sci & Tech Co Ltd

Inventors

• Xia Jing
• Liu Jun
• Xu Binbin
• Shi Kuan
• Li Shupeng
• Qian Zhengyu
• Shi En

Classifications

• G06F8/71 Primary

  Version control (security arrangements therefor G06F21/57); Configuration management · CPC title

• G06F11/3668

  Testing of software · CPC title

• G06N20/00

  Machine learning · CPC title

• G06F9/44505 Primary

  Configuring for program initiating, e.g. using registry, configuration files · CPC title