Systems and methods for independent audit and assessment framework for AI systems

Patent metadata

Field	Value
Publication number	US-12585715-B2
Application number	US-202418942117-A
Country	US
Kind code	B2
Filing date	Nov 8, 2024
Priority date	Nov 8, 2023
Publication date	Mar 24, 2026
Grant date	Mar 24, 2026

Abstract

Example systems and methods for AI capability assessment include query-based assessment of sequential decision making agents (SDMAs) in stochastic settings with minimal assumptions on SDMA internals. In these examples, a new approach is presented for modeling the capabilities of black-box AI systems by using an active learning approach that can effectively interact with the black-box AI systems and learn an interpretable probabilistic model describing the capabilities of the black-box AI systems.

First claim

What is claimed is:

1 . A method for autonomous assessment of sequential decision-making systems in stochastic settings, comprising:
accessing an input A, that is the software and/or integrated hardware constituting an agent of a sequential decision making (SDM) system capable of sequential decision making, or planning, and a simulator Sim that the agent A can interact with to execute its behavior; inducing a system that accepts users' specification of objectives and computes and executes the behavior necessary for achieving them, learning an output model defining capabilities of the SDM system using query-based autonomous capability estimation, including:
defining M as a set of capability models expressible in terms of the set of interpretable concepts or predicates known to the user,
discovering and learning true capabilities of the agent A as a model in M in a manner that clarifies to the user the agent A's true capabilities and the effects of executing them, by posing queries to the agent of the SDM system by:
generating possible capability models for a specific predicate and location combination within M,
utilizing a forward search to generate policy simulation queries for a randomly selected subset of the generated possible capability models, wherein the policy simulation queries output a policy to reach a state where the subset of generated possible capability models provide different responses from each other, and
utilizing responses to the policy simulation queries as data to eliminate possible capability models inconsistent with A's true capabilities from the set of capability models M,

wherein the output model as learned models the set of A's capabilities in stochastic settings.

2 . The method of claim 1 , further comprising measuring a variational distance between the model and a true transition model associated with the SDMA system to evaluate how close the output model is to the true transition model.

3 . The method of claim 1 , further comprising using object-centric predicate representation including an abstraction of environment states to high-level logical states expressible in predicate vocabulary to induce the set of transition models.

4 . The method of claim 1 , wherein the output model as learned is a probabilistic model expressed in terms of the set of predicates and the set of capabilities.

5 . The method of claim 1 , wherein the output model as learned is a non-deterministic model expressed in terms of the set of predicates and the set of capabilities.

6 . The method of claim 1 , wherein the set of predicates are first order logic predicates and the evaluation functions include Boolean evaluation functions.

7 . The method of claim 1 , wherein the output model is learned using input concept vocabulary modeled as binary-valued predicates.

8 . The method of claim 1 , wherein the output model is learned by utilizing syntax and semantics of known SDM model representation language and probabilistic planning definition language.

Assignees

• Srivastava Siddharth
• Verma Pulkit
• Univ Arizona State

Inventors

• Srivastava Siddharth
• Verma Pulkit

Classifications

• G06F16/29

  Geographical information databases · CPC title

• G06F16/243

  Natural language query formulation · CPC title

• G06F40/30

  Semantic analysis · CPC title

• G06F16/953 Primary

  Querying, e.g. by the use of web search engines · CPC title