Large language model architecture to leverage public and private data

Patent metadata

Field	Value
Publication number	US-12505134-B2
Application number	US-202418405168-A
Country	US
Kind code	B2
Filing date	Jan 5, 2024
Priority date	Jan 5, 2024
Publication date	Dec 23, 2025
Grant date	Dec 23, 2025

Abstract

Aspects of the disclosure include methods and systems for an intelligent chat powered by a large language model that leverages both public and private data to answer user questions. An exemplary method includes receiving a user query including natural language input from a user and executing the user query against at least one public data source and at least one private data source. Queries executed against a public source are retrieved using public search indices and queries executed against a private data source are retrieved using user credentials. A query rewrite and a query context including the user query and retrieved information from the public and private data sources are input to a large language model. A response is received from the large language model that includes a natural language answer to the user query and a link to the retrieved information.

First claim

What is claimed is:

1 . A method comprising:
receiving a user query comprising natural language input from a user; executing the user query against at least one public data source and at least one private data source, wherein the executing comprises:
for queries executed against a public data source, retrieving one or more candidate public documents using one or more available public search indices; and
for queries executed against a private data source, retrieving one or more candidate private documents using credentials of the user;
generating a query rewrite by transforming the user query; building a query context comprising the natural language input from the user query and retrieved information from at least one document of the one or more candidate public documents and at least one document of the one or more candidate private documents; providing, to a large language model, a query comprising the query rewrite and the query context; and receiving, from the large language model, a response comprising a natural language answer.

2 . The method of claim 1 , wherein transforming the user query comprises using at least one of synonym expansion, entity recognition, context clarification, question transformation, parameter variation, hierarchical processing, feedback loops, and error handling.

3 . The method of claim 1 , wherein retrieving one or more candidate private documents comprises:
identifying, using the credentials of the user, a list of authorized private search indices for the user; and executing the user query against one or more private search indices within the list of authorized private search indices for the user.

4 . The method of claim 3 , wherein retrieving one or more candidate private documents further comprises retrieving one or more candidate private documents using only the list of authorized private search indices.

5 . The method of claim 1 , further comprising selecting a subset of the candidate public documents and candidate private documents having a highest semantic similarity to the user query.

6 . The method of claim 5 , wherein the response further comprises a link to the retrieved information.

7 . The method of claim 5 , further comprising ranking the selected subset of the candidate public documents and candidate private documents.

8 . The method of claim 7 , wherein ranking the selected subset comprises sorting the selected subset of the candidate public documents and candidate private documents in order of their respective rankings within their respective search indices.

9 . The method of claim 1 , wherein providing the query comprises structuring the query to differentiate between the user query and the retrieved information.

10 . The method of claim 1 , wherein the response visually distinguishes between information obtained from the public data source and information obtained from the private data source.

11 . A system having a memory, computer readable instructions, and one or more processors for executing the computer readable instructions, the computer readable instructions controlling the one or more processors to perform operations comprising:
receiving, at a large language model, a query context comprising retrieved information from at least one document of one or more candidate public documents and one or more candidate private documents; generating, by the large language model, a query rewrite by transforming a user query comprising natural language input; and generating, by the large language model, a response to the user query, the response comprising a natural language answer.

12 . The system of claim 11 , wherein generating the query rewrite comprises mapping the user query to a prompt template.

13 . The system of claim 12 , wherein the prompt template comprises instructions to the large language model to limit the response to directly sourced answers or to respond accordingly if no information is found.

14 . The system of claim 12 , wherein the prompt template distinguishes between the user query and the retrieved information.

15 . The system of claim 11 , wherein the response further comprises a link to the retrieved information.

16 . The system of claim 15 , wherein directing the user device to the retrieved information comprises providing a cached version of the retrieved information to the user device.

17 . A system having a memory, computer readable instructions, and one or more processors for executing the computer readable instructions, the computer readable instructions controlling the one or more processors to perform operations comprising:
receiving, from a client by a large language model, a user query comprising natural language input from a user and credentials of the user; executing, by the large language model, the user query against at least one public data source using one or more public search indices to identify a public document; executing, by the large language model, the user query against at least one private data source using the credentials of the user to access the at least one private data source to identify a private document; and providing, by the large language model to the client, a response comprising a natural language answer to the user query that leverages at least one of the public document and the private document.

18 . The system of claim 17 , wherein the response further comprises a link to at least one of the public document and the private document.

19 . The system of claim 18 , further comprising directing the client to the private document responsive to receiving a selection of the link.

20 . The system of claim 18 , further comprising providing a copy of the private document to the client responsive to receiving a selection of the link.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Olson Jamie
• Xu Wenjin
• Qin Frederick
• Guo Qiwen
• Li Li
• Min Rui
• Li Bing

Classifications

• G06F40/40

  Processing or translation of natural language (natural language analysis G06F40/20; semantic analysis G06F40/30) · CPC title

• G06F40/30

  Semantic analysis · CPC title

• G06F16/955

  using information identifiers, e.g. uniform resource locators [URL] · CPC title

• G06F16/3338 Primary

  Query expansion · CPC title