Constraint-based multi-party image modification

Patent metadata

Field	Value
Publication number	US-2023089790-A1
Application number	US-202117479970-A
Country	US
Kind code	A1
Filing date	Sep 20, 2021
Priority date	Sep 20, 2021
Publication date	Mar 23, 2023
Grant date	—

Abstract

A computer hardware system having an image modification platform includes a hardware processor configured to initiate the following executable operations. A digital image is received by the image modification platform. A plurality of objects within the digital image are detected using an object segmentation engine of the image modification platform. The plurality of objects are classified using an object classification engine of the image modification platform and involve associating a stakeholder, respectively, to at least two of the plurality of objects. A knowledge map for the digital image is generated based upon the classifying and the stakeholders using a knowledge map engine. A set of constraints are identified for each of the plurality of objects based upon the knowledge map and the plurality of stakeholders. A modified digital image to be published is generated by an interactive image generation engine using the sets of constraints.

First claim

What is claimed is:

1 . A computer-implemented process performed by a computer hardware system including an image modification platform, comprising:
receiving, by the image modification platform, a digital image; detecting, using an object segmentation engine of the image modification platform, a plurality of objects within the digital image; classifying, using an object classification engine of the image modification platform, the plurality of objects, the classifying including associating a stakeholder, respectively, to at least two of the plurality of objects; generating, using a knowledge map engine, a knowledge map for the digital image based upon the classifying and the stakeholders, identifying a set of constraints, for each of the plurality of objects based upon the knowledge map and the plurality of stakeholders; generating, by an interactive image generation engine, a modified digital image using the sets of constraints to be published.

2 . The process of claim 1 , wherein
the image modification platform is configured to initiate, based upon the knowledge map, a multimodal communication with the plurality of stakeholders, and the sets of constraints are identified based upon information received by the image modification platform during the multimodal communication.

3 . The process of claim 2 , wherein
at least a portion of the sets of constraints are automatically generated using the information received by the image modification platform during the multimodal communication.

4 . The process of claim 2 , wherein
information provided to the stakeholders during the multimodal communication is based upon the knowledge map.

5 . The process of claim 2 , wherein
a version of the modified digital image is provided to the stakeholders, feedback to the version of the modified digital image is received from the stakeholders and provided during the multimodal communication, the sets of constraints are modified based upon the feedback, and another version of the modified digital image is generated using the modified plurality of constraints.

6 . The process of claim 5 , wherein
the sets of constraints are iteratively modified based upon one or more additional versions of the modified image until a last modified digital image provided to the stakeholders is approved by all of the stakeholders, and the last modified digital image is the modified digital image to be published.

7 . The process of claim 1 , wherein
at least one set of the sets of constraints include a positional constraint, a visual constraint, a privacy constraint, and a neighboring constraint.

8 . The process of claim 1 , wherein
the knowledge map for the digital image maps relationships between objects within the digital image and includes attributes for each of the objects.

9 . A computer hardware system including an image modification platform, comprising:
a hardware processor configured to initiate the following executable operations:
receiving, by the image modification platform, a digital image;
detecting, using an object segmentation engine of the image modification platform, a plurality of objects within the digital image;
classifying, using an object classification engine of the image modification platform, the plurality of objects, the classifying including associating a stakeholder, respectively, to at least two of the plurality of objects;
generating, using a knowledge map engine, a knowledge map for the digital image based upon the classifying and the stakeholders,
identifying a set of constraints, for each of the plurality of objects based upon the knowledge map and the plurality of stakeholders;
generating, by an interactive image generation engine, a modified digital image using the sets of constraints to be published.

10 . The computer hardware system of claim 9 , wherein
the image modification platform is configured to initiate, based upon the knowledge map, a multimodal communication with the plurality of stakeholders, and the sets of constraints are identified based upon information received by the image modification platform during the multimodal communication.

11 . The computer hardware system of claim 10 , wherein
at least a portion of the sets of constraints are automatically generated using the information received by the image modification platform during the multimodal communication.

12 . The computer hardware system of claim 10 , wherein
information provided to the stakeholders during the multimodal communication is based upon the knowledge map.

13 . The computer hardware system of claim 10 , wherein
a version of the modified digital image is provided to the stakeholders, feedback to the version of the modified digital image is received from the stakeholders and provided during the multimodal communication, the sets of constraints are modified based upon the feedback, and another version of the modified digital image is generated using the modified plurality of constraints.

14 . The computer hardware system of claim 13 , wherein
the sets of constraints are iteratively modified based upon one or more additional versions of the modified image until a last modified digital image provided to the stakeholders is approved by all of the stakeholders, and the last modified digital image is the modified digital image to be published.

15 . The computer hardware system of claim 9 , wherein
at least one set of the sets of constraints include a positional constraint, a visual constraint, a privacy constraint, and a neighboring constraint.

16 . The computer hardware system of claim 9 , wherein
the knowledge map for the digital image maps relationships between objects within the digital image and includes attributes for each of the objects.

17 . A computer program product, comprising:
a computer readable storage medium having stored therein program code, the program code, which when executed by a computer hardware system including an image modification platform, causes the computer hardware system to perform:
receiving, by the image modification platform, a digital image;
detecting, using an object segmentation engine of the image modification platform, a plurality of objects within the digital image;
classifying, using an object classification engine of the image modification platform, the plurality of objects, the classifying including associating a stakeholder, respectively, to at least two of the plurality of objects;
generating, using a knowledge map engine, a knowledge map for the digital image based upon the classifying and the stakeholders,
identifying a set of constraints, for each of the plurality of objects based upon the knowledge map and the plurality of stakeholders;
generating, by an interactive image generation engine, a modified digital image using the sets of constraints to be published.

18 . The computer program product of claim 17 , wherein
the image modification platform is configured to initiate, based upon the knowledge map, a multimodal communication with the plurality of stakeholders, the sets of constraints are identified based upon information received by the image modification platform during the multimodal communication, at least a portion of the sets of constraints are automatically generated using the information received by the image modification platform during the multimodal communication, and information provided to the stakeholders during the multimodal communication is based upon the knowledge map.

19 .

Assignees

• IBM

Inventors

• Pandey Diwesh
• MARVANIYA Smitkumar Narotambhai
• Ghosh Subarna
• Ghosh Shiladitya

Classifications

• G06V20/70

  Labelling scene content, e.g. deriving syntactic or semantic representations · CPC title

• G06T7/11 Primary

  Region-based segmentation · CPC title

• G06T2207/30196

  Human being; Person · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06F18/24

  Classification techniques · CPC title