Image processing apparatus and operating method thereof

Patent metadata

Field	Value
Publication number	US-12175678-B2
Application number	US-202217749602-A
Country	US
Kind code	B2
Filing date	May 20, 2022
Priority date	May 20, 2021
Publication date	Dec 24, 2024
Grant date	Dec 24, 2024

Abstract

An image processing apparatus, including a memory configured to store one or more instructions; and at least one processor configured to execute the one or more instructions to: based on a first image and a probability model, optimize an estimated pixel value and estimated gradient values of each pixel of an original image corresponding to the first image, obtain an estimated original image based on the optimized estimated pixel value of the each pixel of the original image, obtain a decontour map based on the optimized estimated pixel value and the estimated gradient values of the each pixel of the original image, and generate a second image by combining the first image with the estimated original image based on the decontour map.

First claim

What is claimed is:

1. An image processing apparatus comprising:
a memory configured to store one or more instructions; and
at least one processor configured to execute the one or more instructions to:
based on a first image and a probability model, optimize an estimated pixel value and estimated gradient values of each pixel of an original image corresponding to the first image,
obtain an estimated original image based on the optimized estimated pixel value of the each pixel of the original image,
obtain a decontour map based on the optimized estimated pixel value and the estimated gradient values of the each pixel of the original image, and
generate a second image by combining the first image with the estimated original image based on the decontour map.

2. The image processing apparatus of claim 1 , wherein the first image includes a true contour which is included in the original image, and a false contour which is not included in the original image, and
wherein the second image is an image from which the false contour is removed.

3. The image processing apparatus of claim 1 , wherein the at least one processor is further configured to execute the one or more instructions to:
obtain, based on the first image, initial values of the estimated pixel value and the estimated gradient values of the each pixel of the original image, and
optimize the estimated pixel value and the estimated gradient values of the each pixel of the original image by updating the estimated pixel value and the estimated gradient values of the each pixel of the original image based on the probability model.

4. The image processing apparatus of claim 3 , wherein the probability model is obtained by modeling a probability that a pixel value of each pixel included in a first region around a first pixel of the first image originates from a first pixel of the original image having the estimated pixel value and the estimated gradient values, and
wherein the at least one processor is further configured to execute the one or more instructions to optimize the estimated pixel value and the estimated gradient values of the first pixel by updating the estimated pixel value and the estimated gradient values of the first pixel such that the probability increases.

5. The image processing apparatus of claim 4 , wherein the at least one processor is further configured to execute the one or more instructions to:
obtain pixel values of pixels included in a second region around the first pixel in the original image, based on the estimated pixel value and the estimated gradient values of the first pixel, and
optimize the estimated pixel value and the estimated gradient values of the first pixel by updating the estimated pixel value and the estimated gradient values of the first pixel such that a loss function determined based on the probability is minimized, and
wherein the probability is represented by a function of a difference between a pixel value of each pixel included in the second region and a pixel value of the each pixel included in the first region.

6. The image processing apparatus of claim 1 , wherein the at least one processor is further configured to execute the one or more instructions to:
obtain a texture map based on the first image and the estimated original image,
obtain a curvature map based on the optimized estimated gradient values of the each pixel of the original image, and
obtain the decontour map based on the texture map and the curvature map.

7. The image processing apparatus of claim 6 , wherein the at least one processor is further configured to execute the one or more instructions to generate the texture map by obtaining a difference image between the first image and the estimated original image and performing filtering on the difference image.

8. The image processing apparatus of claim 6 , wherein the at least one processor is further configured to execute the one or more instructions to obtain the curvature map by computing a curvature of each pixel of the estimated original image based on the optimized estimated gradient values of the each pixel of the original image.

9. The image processing apparatus of claim 6 , wherein the texture map represents a first weight of the each pixel of the original image,
wherein the curvature map represents a second weight of the each pixel of the original image,
wherein the decontour map represents a third weight of the each pixel of the original image, and
wherein the at least one processor is further configured to execute the one or more instructions to obtain the third weight based on the first weight and the second weight of the each pixel of the original image.

10. The image processing apparatus of claim 1 , wherein the at least one processor is further configured to execute the one or more instructions to:
receive a third image comprising a frame image subsequent to the first image,
obtain first estimated information including the estimated pixel value and the estimated gradient values of the each pixel of the original image, which are optimized and subsampled with respect to the first image,
obtain second estimated information by subsampling initial values of an estimated pixel value and estimated gradient values of each pixel of the third image,
obtain difference information between the first image and the third image,
obtain third estimated information based on the first estimated information and the difference information,
obtain fourth estimated information by combining the second estimated information with the third estimated information based on a first probability that the third image originates from the second estimated information and a second probability that the third image originates from the third estimated information,
obtain fifth estimated information by performing optimization on the fourth estimated information based on the subsampled third image, and
generate a fourth image from which a false contour of the third image is removed, based on an estimated pixel value and estimated gradient values included in the fifth estimated information.

11. The image processing apparatus of claim 10 , wherein the at least one processor is further configured to execute the one or more instructions to:
obtain a third probability that a false contour of the first image is included in each pixel of the first image, and
obtain the fourth estimated information by combining the second estimated information and the third estimated information based on the first probability, the second probability, and the third probability.

12. The image processing apparatus of claim 10 , wherein the at least one processor is further configured to execute the one or more instructions to:
obtain an estimated original image of the third image by upscaling the estimated pixel value included in the fifth estimated information,
obtain a texture map based on the third image and the estimated original image of the third image,
obtain a curvature map based on the estimated gradient values included in the fifth estimated information,
upscale the curvature map,
generate a decontour map of the third image based on the texture map and the upscaled curvature map, and
generate the fourth image based on the decontour map of the third image.

13. An operating method of an image processing apparatus, the operating method comprising:
based on a first image and a probability model, optimizing an estimated pixel value and estimated gradient values of each pixel of an original image corresponding to the first image;
obtaining an estimated original image based on the optimized estimated pixel value of the each pixel of the or

Assignees

• Samsung Electronics Co Ltd

Inventors

• Shin Hanul
• Kang Soomin
• Park Jaeyeon
• Song Youngchan
• Ahn Iljun
• Lee Tammy

Classifications

• G06V10/44

  Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components · CPC title

• G06V10/54

  relating to texture · CPC title

• G06T5/50 Primary

  using two or more images, e.g. averaging or subtraction · CPC title

• G06T5/70

  Denoising; Smoothing · CPC title

• G06T5/77

  Retouching; Inpainting; Scratch removal · CPC title