Image processing device for improving details of an image, and operation method of the same

Patent metadata

Field	Value
Publication number	US-12443679-B2
Application number	US-202418427442-A
Country	US
Kind code	B2
Filing date	Jan 30, 2024
Priority date	Dec 4, 2018
Publication date	Oct 14, 2025
Grant date	Oct 14, 2025

Abstract

Provided are an image processing apparatus and an operation method of the image processing apparatus. The image processing apparatus includes a memory storing one or more instructions, and a processor configured to execute the one or more instructions stored in the memory to, by using one or more convolution neural networks, extract target features by performing a convolution operation between features of target regions having same locations in a plurality of input images and a first kernel set, extract peripheral features by performing a convolution operation of features of peripheral regions located around the target regions in the plurality of input images and a second kernel set, and determine a feature of a region corresponding to the target regions in an output image, based on the target features and the peripheral features.

First claim

What is claimed is:

1. An image processing apparatus comprising:
a memory storing one or more instructions; and
a processor configured to execute the one or more instructions stored in the memory,
wherein the processor is configured to, by using one or more convolution neural networks:
extract first features based on features of first regions having same locations in a plurality of input images,
extract second features based on features of peripheral regions located around the first regions in the plurality of input images, and
determine a feature of a region in an output image, corresponding to the first regions, based on the first features and the second features,
wherein the plurality of input images comprise a first input image and a second input image,
wherein the first regions comprise a first region and a second region having same locations in the first input image and the second input image, respectively, and
wherein the peripheral regions comprise a first peripheral region located around the first region in the first input image and a second peripheral region located around the second region in the second input image.

2. The image processing apparatus of claim 1 , wherein the processor is further configured to
extract the first peripheral region in a range where a distance from the first region is within a preset distance, from the first input image, and
extract the second peripheral region in a range where a distance from the second region is within the preset distance, from the second input image.

3. The image processing apparatus of claim 1 , wherein the processor is further configured to
determine a first feature similarity between each of a plurality of regions included in the first input image and the first region, and determine the first peripheral region based on the first feature similarity, and
determine a second feature similarity between each of a plurality of regions included in the second input image and the second region, and determine the second peripheral region based on the second feature similarity.

4. The image processing apparatus of claim 3 , wherein the first peripheral region is a region having a feature most similar to a feature of the first region from among the plurality of regions included in the first input image, and
the second peripheral region is a region having a feature most similar to a feature of the second region from among the plurality of regions included in the second input image.

5. The image processing apparatus of claim 1 , wherein the processor is further configured to
determine a first weight that is applied to the first peripheral region, based on a distance between the first region and the first peripheral region,
determine a second weight that is applied to the second peripheral region, based on a distance between the second region and the second peripheral region, and
extract the second features by applying the first weight and the second weight.

6. The image processing apparatus of claim 5 , wherein the processor is further configured to determine the first weight as a larger value as the first region and the first peripheral region are closer to each other, and determine the second weight as the larger value as the second region and the second peripheral region are closer to each other.

7. The image processing apparatus of claim 1 , wherein the processor is further configured to
determine a first weight that is applied to the first peripheral region, based on a first similarity between a feature of the first region and a feature of the first peripheral region,
determine a second weight that is applied to the second peripheral region, based on a second similarity between a feature of the second region and a feature of the second peripheral region, and
extract the second features by applying the first weight and the second weight.

8. The image processing apparatus of claim 7 , wherein the processor is further configured to determine the first weight as a larger value as the first similarity is larger, and determine the second weight as the larger value as the second similarity is larger.

9. An operation method of an image processing apparatus, the operation method comprising:
extracting first features based on features of first regions having same locations in a plurality of input images;
extracting second features based on features of peripheral regions located around the first regions in the plurality of input images; and
determining a feature of a region in an output image, corresponding to the first regions, based on the first features and the second features,
wherein the plurality of input images comprise a first input image and a second input image,
wherein the first regions comprise a first region and a second region having same locations in the first input image and the second input image, respectively, and
wherein the peripheral regions comprise a first peripheral region located around the first region in the first input image and a second peripheral region located around the second region in the second input image.

10. The operation method of claim 9 , wherein the extracting of the second features comprises:
extracting the first peripheral region in a range where a distance from the first region is within a preset distance, from the first input image; and
extracting the second peripheral region in a range where a distance from the second region is within the preset distance, from the second input image.

11. The operation method of claim 9 , wherein the extracting of the second features comprises:
determining a first feature similarity between each of a plurality of regions included in the first input image and the first region, and determining the first peripheral region based on the first feature similarity; and
determining a second feature similarity between each of a plurality of regions included in the second input image and the second region, and determining the second peripheral region based on the second feature similarity.

12. The operation method of claim 11 , wherein the determining of the first peripheral region comprises determining, as the first peripheral region, a region having a feature most similar to a feature of the first region from among the plurality of regions included in the first input image, and
the determining of the second peripheral region comprises determining, as the second peripheral region, a region having a feature most similar to a feature of the second region from among the plurality of regions included in the second input image.

13. The operation method of claim 9 , wherein the extracting of the second features comprises:
determining a first weight that is applied to the first peripheral region, based on a distance between the first region and the first peripheral region;
determining a second weight that is applied to the second peripheral region, based on a distance between the second region and the second peripheral region; and
extracting the second features by applying the first weight and the second weight.

14. The operation method of claim 13 , wherein the determining the first weight comprises determining the first weight as a larger value as the first region and the first peripheral region are closer to each other, and
wherein the determining the second weight comprises determining the second weight as the larger value as the second region and the second peripheral region are closer to each other.

15. The operation method of claim 9 , further comprising:
determining a first weight that is applied to the first peripheral region, based on a first similarity

Assignees

• Samsung Electronics Co Ltd

Inventors

• Park Jaeyeon
• Ahn Iljun
• Park Yongsup
• Kwak Jaehee
• Lee Tammy
• Cheon Minsu

Classifications

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title

• G06V10/759 Primary

  Region-based matching · CPC title

• G06V10/40

  Extraction of image or video features · CPC title

• G06T7/11 Primary

  Region-based segmentation · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title