Modification and generation of conditional data

Patent metadata

Field	Value
Publication number	US-12443879-B2
Application number	US-202217664239-A
Country	US
Kind code	B2
Filing date	May 20, 2022
Priority date	May 20, 2022
Publication date	Oct 14, 2025
Grant date	Oct 14, 2025

Abstract

A processor may gather raw data comprising a plurality of characteristic data samples of a target user group. The processor may categorize the characteristic data samples into a plurality of user-related classes and triggers. The processor may build an input property graph for each characteristic data sample. The processor may augment the input property graph by a concept of hierarchies. The processor may determine a modification vector from the augmented input property graph. The processor may train an encoder/decoder combination machine-learning system. An embedding vector and a modification vector are used as input for the decoder to build a trained machine-learning generative model.

First claim

What is claimed is:

1. A computer-implemented method for conditional data modification, the method comprising:
gathering raw data comprising a plurality of characteristic data samples of a target user group;
categorizing the characteristic data samples into a plurality of user-related classes and triggers;
building an input property graph for each characteristic data sample, wherein the input property graph comprises data relationships associated with characterial triggers, user identifiers, object identifiers and activity identifiers;
augmenting the input property graph by a concept of hierarchies;
determining a modification vector from the augmented input property graph;
training an encoder/decoder combination machine-learning system comprising a machine learning generative model that is a combined model of an encoder and a decoder, wherein the training comprises:
inputting the characteristic data samples into the encoder to generate an embedding vector;
inputting the embedding vector and the modification vector into the decoder to build the machine-learning generative model,

wherein the machine-learning generative model is configured to output modified data samples relating to the characteristic data samples; and
optimizing the machine-learning generative model using target modified samples as ground truth relating pairwise to the modified data samples; and

receiving, at the trained encoder/decoder combination machine-learning system, inference input data and a conditional input property graph, wherein the conditional input property graph includes a request for a target characterial trigger.

2. The method according to claim 1 , further comprising:
generating new data during the conditional data modification.

3. The method according to claim 1 , wherein the encoder and the decoder are convolutional neural networks (CNNs), and
wherein a number of input nodes of the decoder is greater than a sum of dimensions associated with the embedding vector and the modification vector.

4. The method according to claim 1 , wherein the input property graph is a knowledge graph which is associated with a Resource Description Framework (RDF) nomenclature.

5. The method according to claim 1 , wherein the raw data is a type selected from the group consisting of an image, a text document, an audio stream, a video stream, and an infographic.

6. The method according to claim 1 , wherein the augmenting comprises hierarchically structuring the data relationships.

7. The method according to claim 6 , wherein each node of the augmented input property graph relates to a node embedding, wherein the modification vector is built from a concatenation of the node embeddings, and wherein the modification vector is built from an aggregation of the node embeddings using a graph neural network.

8. The method of claim 1 , further comprising:
predicting, in response to the receiving, output data of a same data representation as the inference input data.

9. The method according to claim 8 , wherein the modification vector associated with the conditional input property graph extends the embedding vector predicted by the encoder to generate an extended embedding vector, wherein the extended embedding vector is input to the decoder to generate the output of the encoder/decoder combination machine-learning system.

10. The method according to claim 8 , wherein a dimension of a vector associated with the augmented input property graph is smaller than a dimension of the embedding vector.

11. A system for conditional data modification, the training-system comprising:
a memory; and
a processor in communication with the memory, the processor being configured to perform operations comprising:
gathering raw data comprising a plurality of characteristic data samples of a target user group;
categorizing the characteristic data samples into a plurality of user-related classes and triggers;
building an input property graph for each characteristic data sample, wherein the input property graph comprises data relationships associated with characterial triggers, user identifiers, object identifiers and activity identifiers;
augmenting the input property graph by a concept of hierarchies, whereby an augmented input property graph is built;
determining a modification vector from the augmented input property graph;
training an encoder/decoder combination machine-learning system comprising a machine learning generative model that is a combined model of an encoder and a decoder, wherein the training comprises:
inputting the characteristic data samples into the encoder to generate an embedding vector;
inputting the embedding vector and the modification vector into the decoder to build the machine-learning generative model, wherein the machine-learning generative model is configured to output modified data samples relating to the characteristic data samples; and
optimizing the machine-learning generative model using target modified samples as ground truth relating pairwise to the modified data samples; and

receiving, at the trained encoder/decoder combination machine-learning system, inference input data and a conditional input property graph, wherein the conditional input property graph includes a request for a target characterial trigger.

12. The system according to claim 11 , further comprising:
generating new data during the conditional data modification.

13. The system according to claim 11 , wherein the encoder and the decoder are convolutional neural networks (CNNs), wherein a number of input nodes of the decoder is equal to a sum of dimensions associated with the embedding vector and the modification vector.

14. The system according to claim 11 , wherein the input property graph is a knowledge graph associated with a Resource Description Framework (RDF) nomenclature.

15. The system according to claim 11 , wherein the raw data is a type selected from the group consisting of an image, a text document, an audio stream, a video stream, and an infographic.

16. The system according to claim 11 , wherein the augmenting comprises hierarchically structuring the data relationships, and wherein the input property graph further comprises data relationships associated with abstract shared objects, action classes, target sentiment identifiers, and target characterial trigger identifiers.

17. The system according to claim 16 , wherein each node of the augmented input property graph relates to a node embedding, and wherein the modification vector is built from a concatenation of the node embeddings.

18. The system of claim 11 , further comprising:
predicting, in response to the receiving, output data of a same data representation as the inference input data.

19. A computer program product for conditional data modification, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processors to perform a function, the function comprising:
gathering raw data comprising a plurality of characteristic data samples of a target user group;
categorizing the characteristic data samples into a plurality of user-related classes and triggers;
building an input property graph for each of the characteristic data samples, wherein the input property graph comprises data relationships associated with characterial triggers, user identifiers, object identifiers and activity identifiers;
augmenting the input property graph by a concept of hierarchies;

Assignees

• IBM

Inventors

• Giovannini Andrea
• Flöther Frederik Frank
• Lustenberger Patrick
• Ocheltree David

Classifications

• G06N3/042

  Knowledge-based neural networks; Logical representations of neural networks · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0475

  Generative networks · CPC title

• G06N5/022

  Knowledge engineering; Knowledge acquisition · CPC title

• G06N20/00 Primary

  Machine learning · CPC title