Characterization for erroneous artificial intelligence outputs

Patent metadata

Field	Value
Publication number	US-12455907-B2
Application number	US-202218061689-A
Country	US
Kind code	B2
Filing date	Dec 5, 2022
Priority date	Dec 5, 2022
Publication date	Oct 28, 2025
Grant date	Oct 28, 2025

Abstract

In some implementations, a device may obtain data indicating reparations issued by an entity that uses artificial intelligence to provide artificial intelligence outputs in connection with users, the reparations being issued for one or more of the artificial intelligence outputs being erroneous. The device may determine, using a machine learning model, an artificial intelligence reparation characterization for the entity. The artificial intelligence reparation characterization determined using the machine learning model may be indicative of an amount of reparations predicted for the entity in connection with uses of artificial intelligence by the entity. The machine learning model may be trained to determine the artificial intelligence reparation characterization based on the data. The device may transmit information indicating the artificial intelligence reparation characterization.

First claim

What is claimed is:

1. A method of characterization for erroneous artificial intelligence outputs, comprising:
obtaining data indicating reparations issued by an entity that uses artificial intelligence to provide artificial intelligence outputs in connection with users, the reparations being issued for one or more of the artificial intelligence outputs being erroneous;
determining, using a machine learning model and based on one or more future entity categories in which the entity is to operate, an artificial intelligence reparation characterization for the entity, wherein the artificial intelligence reparation characterization determined using the machine learning model is indicative of an amount of reparations predicted for the entity in connection with uses of the artificial intelligence by the entity, wherein the machine learning model is trained to determine the artificial intelligence reparation characterization based on the data and the one or more future entity categories, and wherein a first node in an input layer of the machine learning model represents one or more historical reparations and a second node in the input layer of the machine learning model represents one or more user sentiments;
rejecting or approving, using the machine learning model, an application for services based on the artificial intelligence reparation characterization; and
transmitting information indicating the artificial intelligence reparation characterization, wherein the information indicates an approval or a rejection of the application for services.

2. The method of claim 1 , wherein the data is in a blockchain.

3. The method of claim 1 , wherein the data further indicates one or more complaints made against the entity indicating that one or more automated outputs are erroneous.

4. The method of claim 1 , wherein the information includes one or more of a risk category based on the artificial intelligence reparation characterization or a premium amount based on the artificial intelligence reparation characterization.

5. The method of claim 1 , further comprising:
obtaining unstructured data indicating the one or more future entity categories; and
performing natural language processing of the unstructured data to identify the one or more future entity categories,
wherein the machine learning model is trained to determine the artificial intelligence reparation characterization further based on the one or more future entity categories.

6. The method of claim 1 , further comprising:
obtaining unstructured data indicating the one or more user sentiments; and
performing natural language processing of the unstructured data to identify the one or more user sentiments,
wherein the machine learning model is trained to determine the artificial intelligence reparation characterization further based on the one or more user sentiments.

7. The method of claim 1 , wherein the artificial intelligence reparation characterization comprises a score, a classification in a category, or a premium amount.

8. A system for characterization for erroneous artificial intelligence outputs, the system comprising:
one or more memories; and
one or more processors, communicatively coupled to the one or more memories, configured to:
obtain data indicating reparations issued by an entity that uses artificial intelligence to provide artificial intelligence outputs in connection with users, the reparations being issued for one or more of the artificial intelligence outputs being erroneous;
determine, using a machine learning model and based on one or more future entity categories in which the entity is to operate, an artificial intelligence reparation characterization for the entity, wherein the artificial intelligence reparation characterization determined using the machine learning model is indicative of an amount of reparations predicted for the entity in connection with uses of the artificial intelligence by the entity, wherein the machine learning model is trained to determine the artificial intelligence reparation characterization based on the data and the one or more future entity categories, and wherein a first node in an input layer of the machine learning model represents one or more historical reparations and a second node in the input layer of the machine learning model represents one or more user sentiments;
determine, based on the artificial intelligence reparation characterization, information that is to be used to populate a document for the entity;
generate the document with the information populated in the document;
reject or approve, using the machine learning model, an application for services based on the artificial intelligence reparation characterization; and
transmit information indicating the artificial intelligence reparation characterization, wherein the information indicating the artificial intelligence reparation characterization indicates an approval or a rejection of the application for services.

9. The system of claim 8 , wherein the data is in a blockchain.

10. The system of claim 8 , wherein the data further indicates additional reparations issued by one or more other entities.

11. The system of claim 10 , wherein the entity and the one or more other entities are associated with a same entity category.

12. The system of claim 8 , wherein the one or more processors are further configured to:
obtain unstructured data indicating the one or more future entity categories; and
perform natural language processing of the unstructured data to identify the one or more future entity categories,
wherein the machine learning model is trained to determine the artificial intelligence reparation characterization further based on the one or more future entity categories.

13. The system of claim 8 , wherein the one or more processors are further configured to:
obtain unstructured data indicating the one or more user sentiments toward the entity; and
perform natural language processing of the unstructured data to identify the one or more user sentiments,
wherein the machine learning model is trained to determine the artificial intelligence reparation characterization further based on the one or more user sentiments.

14. The system of claim 8 , wherein the one or more processors are further configured to:
receive the application for services, the application for services indicating an entity category associated with the entity,
wherein the one or more processors are configured to obtain the data based on receiving the application for services.

15. The system of claim 8 , wherein the document is a policy document.

16. A non-transitory computer-readable medium storing a set of instructions for characterization for erroneous artificial intelligence outputs, the set of instructions comprising:
one or more instructions that, when executed by one or more processors of a device, cause the device to:
receive an application for services, the application for services indicating an entity category associated with an entity that uses artificial intelligence to provide artificial intelligence outputs in connection with users;
obtain data, in a blockchain, indicating reparations issued by the entity, the reparations being issued for one or more of the artificial intelligence outputs being erroneous;
obtain at least one of first unstructured data indicating one or more future entity categories in which the entity is to operate or second unstructured data indicating one or more user sentiments toward the entity;
perform natural language processing of the at least one of the first unstructured data or the s

Assignees

• Capital One Services Llc

Inventors

• Rafferty Galen
• Sharpe Samuel
• Barr Brian
• Goodsitt Jeremy
• Walters Austin
• Bean Kenny

Classifications

• G06F16/35

  Clustering; Classification · CPC title

• G06F40/56 Primary

  Natural language generation · CPC title

• G06F40/30

  Semantic analysis · CPC title

• G06F16/3329 Primary

  Natural language query formulation · CPC title