Dynamic artificial intelligence agent orchestration using a large language model gateway router

Patent metadata

Field	Value
Publication number	US-12536406-B2
Application number	US-202519279103-A
Country	US
Kind code	B2
Filing date	Jul 24, 2025
Priority date	Apr 11, 2024
Publication date	Jan 27, 2026
Grant date	Jan 27, 2026

Abstract

The systems and methods disclosed herein orchestrate task execution among autonomous (or semi-autonomous) AI agentic models (“agents”) using a gateway router that dynamically coordinates the agents based on prompt characteristics, user context, and/or real-time operational factors. Received inputs (e.g., prompts) are segmented into subcomponents (e.g., sub-queries), which are routed/mapped to candidate agents based on the output parameters of the subcomponent (e.g., performance thresholds, cost thresholds) and operational parameters (e.g., cost, performance metric values, user access restrictions, timing restrictions) of each agent. The gateway router maintains dynamic routing data structures for each agent that are continuously updated based on environmental stimuli (e.g., geo-political stimuli, sensor stimuli, agent stimuli). For example, the gateway router causes agents to dynamically switch between rule engines identified by the routing tables in response to detecting environmental stimuli. Responses from the candidate agents are aggregated into an output that is responsive to the input.

First claim

We claim:

1 . A non-transitory computer-readable storage medium comprising instructions for orchestrating a plurality of autonomous artificial intelligence (AI) agents to generate a personalized response stored thereon, wherein the instructions when executed by at least one data processor of a system, cause the system to:
receive, from a computing device, an output generation request comprising a prompt for generation of an output using one or more AI agents of the plurality of AI agents communicatively connected to a large language model (LLM) set,
wherein each AI agent is associated with a specific routing matrix that identifies a computer-executable task set used to generate a response, and
wherein each computer-executable task is configured to be autonomously executed by the AI agent on a set of software applications in response to satisfaction of a condition set;
segment, using the LLM set, the prompt into a plurality of sub-queries, each sub-query sharing a common output parameter set that identifies one or more of:
a user type associated with the prompt, a timestamp of receiving the output generation request, an output modality, a performance metric threshold, or a system resource usage threshold;
determine an operational parameter set of each AI agent that defines one or more of:
at least one user type authorized to use the AI agent, a range of timestamps associated with the AI agent, at least one output modality of responses generated by the AI agent, at least one performance metric values, or at least one resource usage values;
for each sub-query of the plurality of sub-queries, identify, using the LLM set, a candidate agent from the plurality of AI agents by comparing (a) the output parameter set of the sub-query with (b) the operational parameter set of each AI agent within the plurality of AI agents; for each identified candidate agent of each sub-query:
select, using the LLM set, one or more computer-executable tasks from the computer-executable task set identified by a respective routing matrix of the candidate agent, wherein each of the one or more computer-executable tasks are selected based on the sub-query satisfying a respective condition set of the computer-executable task, and
autonomously execute, using the identified candidate agent, the selected one or more computer-executable tasks to generate an agent-specific response set responsive to the sub-query; and
using the LLM set, aggregate each respective agent-specific response set of each respective candidate agent of each sub-query into an overall response set that is responsive to the prompt of the output generation request.

2 . The non-transitory computer-readable storage medium of claim 1 , wherein the operational parameter set defines the at least one resource usage value, and wherein the instructions further cause the system to:
allocate a subset of available computational resources to process the sub-query based on the one or more resource usage values of the identified candidate agent.

3 . The non-transitory computer-readable storage medium of claim 1 , wherein one or more routing matrixes indicate one or more of: a knowledge source used by a respective AI agent or a model used by the respective AI agent.

4 . The non-transitory computer-readable storage medium of claim 1 , wherein the instructions further cause the system to:
detect a change in one or more environmental signals using the LLM set; and dynamically modify the routing matrix of one or more AI agents based on the detected change in the one or more environmental signals.

5 . The non-transitory computer-readable storage medium of claim 1 , wherein the instructions further cause the system to:
compare the prompt against a database of previous queries,
wherein one or more identified candidate agents are identified based on the comparison.

6 . The non-transitory computer-readable storage medium of claim 1 , wherein the plurality of AI agents is organized in a hierarchal architecture, and
wherein the hierarchal architecture includes a (a) general-purpose agent at a first level of the hierarchal architecture and (b) multiple specialized sub-agents at a second level.

7 . The non-transitory computer-readable storage medium of claim 1 , wherein the instructions further cause the system to:
obtain a feedback set for one or more agent-specific response sets; generating a modification set to modify one or more of: (a) the one or more computer-executable tasks of a respective candidate agent or (b) a sequence of the one or more computer-executable tasks of the respective candidate agent; and transmitting the modification set to the respective candidate agent.

8 . A method for orchestrating an autonomous artificial intelligence (AI) agent set to generate a personalized response, the method comprising:
obtain an output generation request comprising an input for generation of an output using one or more AI agents of the AI agent set communicatively connected to an AI model set,
wherein each AI agent is associated with a specific routing data structure that identifies a computer-executable task set configured to be autonomously executed by the AI agent in response to satisfaction of a condition set;
for each portion of the input, identify, using the AI model set, a candidate agent set from the AI agent set by comparing (a) one or more output parameters of the portion with (b) one or more operational parameters of each AI agent within the AI agent set; for one or more identified candidate agents of each portion:
select, using the AI model set, one or more computer-executable tasks from the computer-executable task set identified by a respective routing data structure of the one or more identified candidate agents, wherein each of the one or more computer-executable tasks are selected based on the portion satisfying a respective condition set of the computer-executable task, and
autonomously execute, using the one or more identified candidate agents, the selected one or more computer-executable tasks to generate an agent-specific response set responsive to the portion; and
using the AI model set, aggregate each respective agent-specific response set of each respective candidate agent set of each portion into an overall response set that is responsive to the input of the output generation request.

9 . The method of claim 8 , wherein the method further comprises:
updating the routing data structure in response to at least one of: (a) a detected change in system load, (b) a change in user context, (c) a change in external environmental signals, or (d) a performance metric associated with the AI agent.

10 . The method of claim 8 , wherein the AI agent set comprises a validation agent configured to validate updates to a knowledge base accessed by one or more agents in the AI agent set, and wherein the method further comprises:
receiving, by the validation agent, a proposed update to the knowledge base, initiating a computer-implemented workflow using the validation agent to evaluate the proposed update against an update criteria set, and responsive to determining satisfaction of the proposed update with the update criteria set, applying the proposed update to the knowledge base.

11 . The method of claim 8 , further comprising:
exposing an application programming interface (API) registry identifying the AI agent set, wherein the API registry is accessible by one or more AI models of the AI model set.

12 . The method of claim 8 , wherein at least one AI model in the AI model set is a Large Language Model (LLM).

13 . The method of claim 8 ,
wherein each AI agent is

Assignees

• Citibank Na

Inventors

• Myers James
• Bhat Ganesh Prasad
• Deb Sourabh
• Engelbrecht Jason
• Wang Zheyu
• Jin Haolin

Classifications

• G06N3/042 Primary

  Knowledge-based neural networks; Logical representations of neural networks · CPC title

• G06F2209/5019

  Workload prediction · CPC title

• G06F16/90332

  Natural language query formulation or dialogue systems · CPC title

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title