Enhancing images in text documents

Patent metadata

Field	Value
Publication number	US-12079912-B2
Application number	US-202218054162-A
Country	US
Kind code	B2
Filing date	Nov 10, 2022
Priority date	Nov 10, 2022
Publication date	Sep 3, 2024
Grant date	Sep 3, 2024

Abstract

Images placed in documents are enhanced based on the context in which the image is used. Context is determined according to document-specific indicators such as nearby text, headings, titles, and tables of content. A generative adversarial network (GAN) modifies the image according to the context to selectively emphasize relevant components of the image, which may include erasing or deleting irrelevant components. Relevant general-purpose images may be retrieved for use in the document and may be selectively enhanced according to usage of the general-purpose image in a given document.

First claim

What is claimed is:

1. A computer-implemented method comprising:
identifying, in a document, an image placed at a location within the document and a set of text proximate the location of the image;
determining, by natural language processing, a meaning of the set of text;
determining a usage context for the image based on the meaning of the set of text, the usage context being a purpose for placing the image at the location within the document;
identifying a set of features in the image;
selecting, from the set of features, a sub-set of features as being related to the usage context; and
determining an enhancement action that emphasizes the selected sub-set of features over remaining features in the set of features.

2. The computer-implemented method of claim 1 , further comprising:
monitoring the document during an editing process; and
detecting placement of the image in the document,
wherein:
the determining the usage context is performed responsive to the detecting placement.

3. The computer-implemented method of claim 1 , further comprising:
analyzing the set of features with respect to the meaning of the set of text to determine a relevance level of each feature to the meaning of the set of text; and
ranking a first feature as being more relevant than a second feature, the first and second features identified within the image;
wherein:
selecting the sub-set of features includes selecting the first feature, the first feature being related to the usage context by being more relevant than the second feature.

4. The computer-implemented method of claim 3 , further comprising:
responsive to the first feature being of high relevance, modifying the image to emphasize the first feature, the modifying including resizing the first feature in the document and repositioning associated text in the image.

5. The computer-implemented method of claim 4 , wherein the modifying further includes increasing resolution of the associated text in the image.

6. The computer-implemented method of claim 1 , further comprising:
modifying the sub-set of features according to the enhancement action by:
inputting the image and the set of features into a generative adversarial network (GAN) to take the enhancement action.

7. The computer-implemented method of claim 1 , further comprising:
identifying the image for placement into the document based on assigned topic from an image repository of general-purpose images; and
presenting the identified image to a user for use in the document.

8. A computer program product comprising a computer-readable storage medium having a set of instructions stored therein which, when executed by a processor, causes the processor to perform a method comprising:
identifying, in a document, an image embedded at a location within the document and a set of text proximate the location of the image;
determining, by natural language processing, a meaning of the set of text;
determining a usage context for the image based on the meaning of the set of text, the usage context being a purpose for placing the image at the location within the document;
identifying a set of features in the image;
selecting, from the set of features, a sub-set of features as being related to the usage context; and
determining an enhancement action that emphasizes the selected sub-set of features over remaining features in the set of features.

9. The computer program product of claim 8 , further comprising:
monitoring the document during an editing process; and
detecting placement of the image in the document,
wherein:
the determining the usage context is performed responsive to the detecting placement.

10. The computer program product of claim 8 , further comprising:
analyzing the set of features with respect to the meaning of the set of text to determine a relevance level of each feature to the meaning of the set of text; and
ranking a first feature as being more relevant than a second feature, the first and second features identified within the image;
wherein:
selecting the sub-set of features includes selecting the first feature, the first feature being related to the usage context by being more relevant than the second feature.

11. The computer program product of claim 10 , further comprising:
responsive to the first feature being of high relevance, modifying the image to emphasize the first feature, the modifying including resizing the first feature in the document and repositioning associated text in the image.

12. The computer program product of claim 11 , wherein the modifying further includes increasing resolution of the associated text in the image.

13. The computer program product of claim 8 , further comprising:
modifying the sub-set of features according to the enhancement action by:
inputting the image and the set of features into a generative adversarial network (GAN) to take the enhancement action.

14. The computer program product of claim 8 , further comprising:
identifying the image for placement into the document based on assigned topic from an image repository of general-purpose images; and
presenting the identified image to a user for use in the document.

15. A computer system comprising:
a processor set; and
a computer readable storage medium;
wherein:
the processor set is structured, located, connected, and/or programmed to run program instructions stored on the computer readable storage medium; and
the program instructions which, when executed by the processor set, cause the processor set to perform a method comprising:
identifying, in a document, an image embedded at a location within the document and a set of text proximate the location of the image;
determining, by natural language processing, a meaning of the set of text;
determining a usage context for the image based on the meaning of the set of text, the usage context being a purpose for placing the image at the location within the document;
identifying a set of features in the image;
selecting, from the set of features, a sub-set of features as being related to the usage context; and
determining an enhancement action that emphasizes the selected sub-set of features over remaining features in the set of features.

16. The computer system of claim 15 , further comprising:
monitoring the document during an editing process; and
detecting placement of the image in the document,
wherein:
the determining the usage context is performed responsive to the detecting placement.

17. The computer system of claim 15 , further comprising:
analyzing the set of features with respect to the meaning of the set of text to determine a relevance level of each feature to the meaning of the set of text; and
ranking a first feature as being more relevant than a second feature, the first and second features identified within the image;
wherein:
selecting the sub-set of features includes selecting the first feature, the first feature being related to the usage context by being more relevant than the second feature.

18. The computer system of claim 17 , further comprising:
responsive to the first feature being of high relevance, modifying the image to emphasize the first feature, the modifying including resizing the first feature in the document, repositioning associated text in the image, and increasing resolution of the associated text in the image.

19. The computer system of claim 15 , further comprising:
modifying the sub-set of features according to the enhancement action by:
inputting the image and t

Assignees

• IBM

Inventors

• Mene Atul
• Keen Martin G
• Rakshit Sarbajit K
• Agrawal Tushar

Classifications

• G06T5/00

  Image enhancement or restoration · CPC title

• G06F18/211

  Selection of the most significant subset of features · CPC title

• G06V30/18

  Extraction of features or characteristics of the image · CPC title

• G06V30/413

  Classification of content, e.g. text, photographs or tables · CPC title

• G06F40/20 Primary

  Natural language analysis (semantic analysis of natural language G06F40/30) · CPC title