Object stitching image generation

Patent metadata

Field	Value
Publication number	US-12400289-B2
Application number	US-202117245191-A
Country	US
Kind code	B2
Filing date	Apr 30, 2021
Priority date	Apr 30, 2021
Publication date	Aug 26, 2025
Grant date	Aug 26, 2025

Abstract

A method includes receiving, by a computing device, concepts of a domain; determining, by the computing device, objects relevant to the concepts; generating, by the computing device, a new image by stitching the relevant objects together; determining, by the computing device, whether the new image is accurate or inaccurate; and in response to determining the new image is inaccurate, propagating, by the computing device, the inaccurate new image back to a convolutional neural network (CNN).

First claim

What is claimed is:

1. A method, comprising:
receiving, by a computing device, a user input comprising concepts of a domain;
determining, by the computing device, objects relevant to the concepts, wherein the relevant objects are not included in the user input;
generating, by the computing device, a new image by stitching the relevant objects together;
generating, by the computing device using a generative adversarial network (GAN), scene graphs to connect the relevant objects to the concepts of the domain;
determining, by the computing device, whether the new image is accurate or inaccurate using the scene graphs generated by the GAN; and
in response to determining the new image is inaccurate, propagating, by the computing device, the inaccurate new image back to a convolutional neural network (CNN).

2. The method of claim 1 , further comprising, in response to determining the new image is accurate, labeling, by the computing device, the accurate new image as a real image, wherein the labeling comprises a descriptor for the concepts of the domain, the domain, and the relevant objects related to the accurate new image; and
storing the label in a knowledge base.

3. The method of claim 1 , wherein the determining the relevant objects include using a concatenation layer of the CNN.

4. The method of claim 3 , wherein the concatenation layer links the concepts together with the relevant objects using domain knowledge.

5. The method of claim 1 , wherein the stitching the objects together includes overlapping the relevant objects so that a field of view of each relevant object overlaps to generate an image with a wider field of view wider than the field of view of each object.

6. The method of claim 1 , further comprising receiving the new image at the GAN and determining whether the new image is accurate or inaccurate.

7. The method of claim 6 , wherein the GAN includes a generator and a discriminator.

8. The method of claim 7 , wherein the determining the new image is accurate or inaccurate includes training the discriminator using the scene graphs from accurate images.

9. The method of claim 8 , further comprising verifying, by the computing device, accuracy of the scene graphs by applying a knowledge base to the scene graphs.

10. The method of claim 1 , wherein the new image is an existing image enriched by the relevant objects and wherein the relevant objects are selected based on a link between the relevant objects and the concepts of the domain.

11. The method of claim 1 , wherein the computing device includes software provided as a service in a cloud environment.

12. A computer program product comprising one or more computer readable storage media having program instructions collectively stored on the one or more computer readable storage media, the program instructions executable to:
receive a user input comprising concepts of a domain;
determine objects relevant to the concepts, wherein the relevant objects are not included in the user input;
generate a new image by stitching the relevant objects together;
determine whether the new image is accurate or inaccurate; and
in response to determining the new image is accurate, label the new image as an accurate new image.

13. The computer program product of claim 12 , wherein a convolutional neural network (CNN) receives the concepts and wherein the relevant objects are selected from a plurality of sources.

14. The computer program product of claim 13 , wherein CNN includes a convolutional layer and the program instructions are executable to filter out less relevant objects with respect to the concepts using a centrality value within the convolutional layer.

15. The computer program product of claim 12 , wherein the program instructions are executable to automatically receive the concepts using computer vision.

16. A system comprising:
a processor, a computer readable memory, one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the program instructions executable to:
receive a user input comprising concepts of a domain;
determine objects relevant to the concepts, wherein the relevant objects are not included in the user input;
generate a new image by stitching the relevant objects together;
apply scene graphs to the new image; and
in response to determining the new image does not match the scene graphs, propagate the new image back to a convolutional neural network (CNN).

17. The system of claim 16 , wherein the CNN includes a subsampling layer which filters out less relevant objects with respect to the concepts.

18. The system of claim 17 , wherein the subsampling layer filters out less relevant objects by down-sampling.

19. The system of claim 18 , wherein the program instructions are further executable to arrange an output of the subsampling layer as a vector.

20. The computer program product of claim 14 , wherein the filtering out less relevant objects comprises down-sampling feature maps of the convolutional later by pooling weights of objects within the feature maps.

Assignees

• IBM

Inventors

• Goyal Prateek
• Nagar Seema
• Bhide Manish Anand
• Dey Kuntal

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/094

  Adversarial learning · CPC title

• G06N3/042

  Knowledge-based neural networks; Logical representations of neural networks · CPC title

• G06N3/0475

  Generative networks · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title