Image retargeting method and device

Patent metadata

Field	Value
Publication number	US-11508035-B2
Application number	US-202117333900-A
Country	US
Kind code	B2
Filing date	May 28, 2021
Priority date	Dec 29, 2020
Publication date	Nov 22, 2022
Grant date	Nov 22, 2022

Abstract

A method for controlling an electronic device is provided. The method includes obtaining an intermediate image by preprocessing an original image, obtaining a saliency feature map of the intermediate image by performing semantic saliency analysis on the intermediate image, performing adaptability calculation according to the saliency feature map and a retargeted target equipment condition, and determining a retargeting mode of the original image according to the result of the adaptability calculation, and performing retargeting processing on the original image according to the determined mode.

First claim

What is claimed is:

1. A method for controlling an electronic device, the method comprising:
obtaining an intermediate image by preprocessing an original image;
obtaining a saliency feature map of the intermediate image by performing semantic saliency analysis on the intermediate image;
performing an adaptability calculation according to the saliency feature map and a retargeted target equipment condition; and
determining a retargeting mode of the original image according to a result of the adaptability calculation, and performing retargeting processing on the original image according to the determined mode,
wherein the performing of the adaptability calculation further comprises, based on proportions of the original image and a target image being consistent, determining that the original image is adaptable.

2. The method of claim 1 , wherein the performing of the adaptability calculation according to the saliency feature map and the retargeted target equipment condition comprises:
based on semantic saliency of the intermediate image being detected, determining that the original image is adaptable; and
based on a distortion caused by preset deformation being within an acceptable range, determining that the original image is adaptable.

3. The method of claim 1 , wherein the determining of the retargeting mode of the original image according to the result of the adaptability calculation comprises at least one of:
based on the adaptability calculation result indicating that the original image is adaptable, determining to perform retargeting processing by directly adjusting a resolution ratio of the original image; or
based on the adaptability calculation result indicating that the original image is not adaptable, selecting the retargeting mode according to the saliency feature map obtained by the semantic saliency analysis, a constraint condition determined by the retargeted equipment condition and a preset neural network model for selecting a retargeting strategy.

4. The method of claim 1 , wherein the semantic saliency analysis comprises at least one of: a human face detection, a character detection, a human body detection, an object detection or a relevance detection.

5. The method of claim 1 , wherein the adaptability calculation comprises at least one of: a width-to-height ratio matching, a saliency calculation, a target equipment analysis, a visual attention calculation or a distortion calculation.

6. The method of claim 1 , wherein the obtaining the intermediate image by preprocessing the original image comprises performing isomorphic processing based on a color and a resolution of the original image.

7. An electronic device comprising:
a memory; and
a processor comprising a preprocessing circuitry, a semantic saliency analysis circuitry, an adaptability calculation circuitry and a retargeting circuitry,
wherein the processor configured to:
obtain, through the preprocessing circuitry, an intermediate image by preprocessing an original image,
obtain, through the semantic saliency analysis circuitry, a saliency feature map of the intermediate image by performing semantic saliency analysis on the intermediate image,
perform, through the adaptability calculation circuitry, an adaptability calculation according to the saliency feature map and a retargeted target equipment condition, and
determine, through the retargeting circuitry, a retargeting mode of the original image according to a result of the adaptability calculation, and perform retargeting processing on the original image according to the determined mode,
wherein the adaptability calculation circuitry is further configured to, based on proportions of the original image and a target image being consistent, determine that the original image is adaptable.

8. The electronic device of claim 7 , wherein the adaptability calculation circuitry is further configured to:
based on semantic saliency of the intermediate image being detected, determine that the original image adaptable; and
based on a distortion caused by preset deformation being within an acceptable range, determine that the original image is adaptable.

9. The electronic device of claim 7 , wherein the retargeting circuitry is further configured to:
based on the adaptability calculation result indicating that the original image is adaptable, determine to perform retargeting processing by directly adjusting a resolution ratio of the original image; and
based on the adaptability calculation result indicating that the original image is not adaptable, select the retargeting mode according to the saliency feature map obtained by the semantic saliency analysis, a constraint condition determined by the retargeted equipment condition and a preset neural network model for selecting a retargeting strategy.

10. The electronic device of claim 7 , wherein the semantic saliency analysis comprises at least one of: a human face detection, a character detection, a human body detection, an object detection or a relevance detection.

11. The electronic device of claim 7 , wherein the adaptability calculation comprises at least one of: a width-to-height ratio matching, a saliency calculation, a target equipment analysis, a visual attention calculation or a distortion calculation.

12. The electronic device of claim 7 , wherein the preprocessing circuitry is further configured to obtain the intermediate image by performing isomorphic processing based on a color and a resolution of the original image.

13. The electronic device of claim 7 , wherein the adaptability calculation result indicates that the original image is adaptable based on the adaptability calculation result being greater than a set threshold.

14. The electronic device of claim 9 , wherein the constraint condition comprises at least one of minimum distortion introduction, associated information preservation, or an unchangeable region.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Yu Jiang
• Chen Youxin
• Wu Longhai
• Chen Jie

Classifications

• G06V10/82

  using neural networks · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T3/4092 Primary

  Image resolution transcoding, e.g. by using client-server architectures · CPC title

• G06T7/0002

  Inspection of images, e.g. flaw detection · CPC title

• G06V40/10

  Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands · CPC title