Research activities through conversational user experience using multi-modal large pretrained models

Patent metadata

Field	Value
Publication number	US-12505308-B2
Application number	US-202318212838-A
Country	US
Kind code	B2
Filing date	Jun 22, 2023
Priority date	Jun 22, 2023
Publication date	Dec 23, 2025
Grant date	Dec 23, 2025

Abstract

The present disclosure relates to methods and systems for using large language models to support research activities. The methods and systems include a copilot engine that creates input prompts to provide to the large language model to use in generating responses to input messages. The copilot engine infers an intent of the input messages and sends the intent with the input message in the input prompt to the large language model. The large language model generates different types of responses for different intents.

First claim

What is claimed is:

1 . A method, comprising:
inferring an intent of an input message that starts a chat session with a large language model; providing, to the large language model, a prompt with the input message and the intent; selecting, by the large language model, artificial intelligence (AI) models from a list of available AI models based on the intent of the input message using model information that identifies parameters of the AI models and functions the AI models perform; receiving, from the large language model, a response with a plan with natural language text in response to the input message, wherein different types of responses are provided by the large language model for different intents and the plan includes a plurality of steps where each step includes a reasoning chain with a description of the AI models selected for use by the large language model for each step; outputting the response; receiving a modification to the reasoning chain adding an AI model to the AI models for a step in the plan; and executing, by the large language model, the plan using the AI models, wherein the AI models ingest data and provide knowledge representations of the data.

2 . The method of claim 1 , wherein the intent is a multistep plan for a query in the input message and the type of response provided by the large language model is a plan with steps for the multistep plan.

3 . The method of claim 2 , wherein the multistep plan is to build a data flow or execute a data operation.

4 . The method of claim 2 , wherein the large language model:
breaks the query in the input message into the steps; generates the plan with the steps for the multistep plan; and outputs the response with the plan and an explanation for each step.

5 . The method of claim 4 , wherein the large language model uses a combination of artificial intelligence (AI) models and formulas to generate a reasoning chain for the steps of the plan.

6 . The method of claim 5 , further comprising:
receiving feedback for the plan with a modification to the plan, wherein the modification is a removal of an AI model or formula, an addition of a formula, a removal of a step, an addition of a step, selecting a different data source for the plan, or editing a step.

7 . The method of claim 6 , wherein the response includes the modification to the plan.

8 . The method of claim 4 , further comprising:
receiving approval for the plan; creating, in response to approval for the plan, pages corresponding to the steps in the plan; and storing, in a chat session history, the pages, wherein the pages provide information for the plan.

9 . The method of claim 8 , further comprising:
receiving an additional page with a formula or AI model; and adding the additional page to the chat session history.

10 . The method of claim 8 , wherein the large language model uses the chat session history in combination with the intent in preparing the response to subsequent input messages.

11 . The method of claim 1 , wherein the intent is seeking an answer for a scientific query in the input message and the type of response provided by the large language model is an answer to the scientific query.

12 . The method of claim 11 , wherein the large language model:
retrieves relevant data based on a relevance score from a grounded dataset for answering the scientific query; and outputs the response with the relevant data with reference links identifying a source of the relevant data.

13 . The method of claim 12 , wherein the large language model ranks the relevant data in an order and provides a top portion of the relevant data in the response.

14 . The method of claim 12 , wherein the grounded dataset includes publicly available data sources and private data sources.

15 . The method of claim 1 , wherein the intent is asking a support question and the type of response provided by the large language model is an answer to the support question with information obtained from a user manual or support guide for providing the answer.

16 . The method of claim 1 , further comprising:
creating a page corresponding to the response, wherein the page provides information for the response; and storing, in a chat session history, the page, wherein the large language model uses the chat session history in combination with the intent in preparing the response to subsequent input messages.

17 . A device, comprising:
a memory to store data and instructions; and a processor operable to communicate with the memory, wherein the processor is operable to:
infer an intent of an input message that starts a chat session with a large language model;
provide, to the large language model, a prompt with the input message and the intent;
select, by the large language model, artificial intelligence (AI) models from a list of available AI models based on the intent of the input message using model information that identifies parameters of the AI models and functions the AI models perform;
receive, from the large language model, a response with a plan with natural language text in response to the input message, wherein different types of responses are provided by the large language model for different intents and the plan includes a plurality of steps where each step includes a reasoning chain with a description of the AI models selected for use by the large language model for each step;
output the response;
receive a modification to the reasoning chain adding an AI model to the AI models for a step in the plan; and
execute, by the large language model, the plan using the AI models, wherein the AI models ingest data and provide knowledge representations of the data.

18 . The device of claim 17 , wherein the intent is a multistep plan for a query in the input message and the type of response provided by the large language model is a plan with steps and the steps include a combination of artificial intelligence (AI) models and formulas to generate a reasoning chain for the steps.

19 . The device of claim 17 , wherein the intent is seeking an answer for a scientific query in the input message and the type of response provided by the large language model is an answer to the scientific query from a grounded dataset with a link identifying a source of the answer.

20 . The device of claim 17 , wherein the intent is asking a support question and the type of response provided by the large language model is an answer to the support question with
information obtained from a user manual or support guide for providing the answer.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Abraham Robin
• Du Liang
• Raja Fahimeh
• Wang Wenhan
• Stewart Dustin James
• Patnaik Lipsa
• Long Stuart Richard
• Earnheart Timothy
• Gammon Sam Daniel
• Arozarena Valladare Sacha
• Singer Jedediah Miller
• Dantas Henrique

Classifications

• G06F16/3329

  Natural language query formulation · CPC title

• G06F16/3328

  using graphical result space presentation or visualisation · CPC title

• G06F16/3344

  using natural language analysis · CPC title

• G06F40/131

  Fragmentation of text files, e.g. creating reusable text-blocks; Linking to fragments, e.g. using XInclude; Namespaces · CPC title

• G06F40/216

  using statistical methods · CPC title