Image processing method for performing color migration to correction image according to reference image, and obtaining first scratch image and/or second scratch image so as to obtain scratch repairing image

Patent metadata

Field	Value
Publication number	US-12198310-B2
Application number	US-202117764458-A
Country	US
Kind code	B2
Filing date	Mar 22, 2021
Priority date	Mar 22, 2021
Publication date	Jan 14, 2025
Grant date	Jan 14, 2025

Abstract

An image processing method and an image processing device are provided. The image processing method includes performing at least one of a scratch repairing step, a dead point repairing step, a denoising step and a color cast correcting step on a to-be-processed video frame. According to the embodiments of the present disclosure, it is able to repair a scratch and a dead point, remove a noise and/or correct color cast for a video frame, thereby to improve a display effect of the video frame.

First claim

What is claimed is:

1. An image processing method, characterized by comprising:
performing at least one of following steps on a to-be-processed video frame:
a scratch repairing step of subjecting the to-be-processed video frame to scratch removal to obtain a first image, performing variance calculation on the to-be-processed video frame and the first image to obtain a variance image, processing the variance image to obtain a scratch image where merely a scratch is reserved, and obtaining a scratch-repaired image in accordance with the to-be-processed video frame and the scratch image;
a dead point repairing step of obtaining consecutive N 1 video frames, the N 1 video frames comprising the to-be-processed video frame, at least one video frame before the to-be-processed video frame and at least one video frame after the to-be-processed video frame, filtering the to-be-processed video frame in accordance with the at least one video frame before the to-be-processed video frame and at least one video frame after the to-be-processed video frame to obtain a dead-point-repaired image, and subjecting the dead-point-repaired image to artifact removal in accordance with the at least one video frame before the to-be-processed video frame and the at least one video frame after the to-be-processed video frame to obtain an artifact-repaired image, N 1 being a positive integer greater than or equal to 3;
a denoising step of denoising the to-be-processed video frame through a denoising network, wherein the denoising network is obtained through following steps: obtaining a target non-movement mask in accordance with consecutive N 2 video frames comprising a to-be-denoised video frame, and training a to-be-trained denoising network in accordance with the N 2 video frame and the target non-movement mask to obtain the denoising network, N 2 being a positive integer greater than or equal to 2; and
a color cast correcting step of determining respective target color cast values of RGB channels of the to-be-processed video frame, performing color balance adjustment on the to-be-processed video frame in accordance with the target color cast values to obtain a first corrected image, and performing color migration on the first corrected image in accordance with a reference image to obtain a second corrected image.

2. The image processing method according to claim 1 , wherein the subjecting the to-be-processed video frame to the scratch removal comprises:
performing median filtration on the to-be-processed video frame in accordance with at least one of a type of a filter and a type of the scratch in the to-be-processed video frame to obtain an image where the scratch has been removed.

3. The image processing method according to claim 2 , wherein the performing median filtration on the to-be-processed video frame in accordance with at least one of the type of the filter and the type of the scratch in the to-be-processed video frame comprises:
performing a respective preprocessing on the to-be-processed video frame in accordance with the type of the filter and the type of the scratch in the to-be-processed video frame, and performing median filtration on the to-be-processed video frame, wherein when the median filter in the horizontal direction is adopted and the scratch in the to-be-processed video frame is a non-vertical scratch, the to-be-processed video frame is rotated to convert the scratch into a vertical scratch; when the median filter in the vertical direction is adopted and the scratch in the to-be-processed video frame is a non-horizontal scratch, the to-be-processed video frame is rotated to convert the scratch into a horizontal scratch;
or
selecting a corresponding type of a filter in accordance with the type of the scratch in the to-be-processed video frame, to perform median filtration on the to-be-processed video frame, wherein when the scratch in the to-be-processed video frame is a vertical scratch, the median filtration is performed on the to-be-processed video frame through a median filter in a horizontal direction; when the scratch in the to-be-processed video frame is a horizontal scratch, the media filtration is performed on the to-be-processed video frame through a median filter in a vertical direction.

4. The image processing method according to claim 1 , wherein
the subjecting the to-be-processed video frame to scratch removal comprises: performing median filtration on the to-be-processed video frame through a median filter having a size of 1*k and/or k*1;
prior to subjecting the to-be-processed video frame to the scratch removal, the image processing method further comprises:
increasing a value of k of the median filter gradually from a predetermined value, and performing median filtration on the to-be-processed video frame to obtain a second image;
determining a final value of k in accordance with a filtering effect of the second image.

5. The image processing method according to claim 1 , wherein the performing variance calculation on the to-be-processed video frame and the first image to obtain the variance image comprises:
performing the variance calculation on the to-be-processed video frame and the first image to obtain a first variance image and/or a second variance image, wherein the first variance image is obtained through subtracting the first image from the to-be-processed video frame, and the second variance image is obtained through subtracting the to-be-processed video frame from the first image;
the processing the variance image to obtain the scratch image where merely the scratch is reserved comprises: processing the first variance image to obtain a first scratch image where merely the scratch is reserved, and/or processing the second variance image to obtain a second scratch image where merely the scratch is reserved;
the obtaining the scratch-repaired image in accordance with the to-be-processed video frame and the scratch image comprises: performing operation on the to-be-processed video frame, the first scratch image and/or the second scratch image to obtain the scratch-repaired image,
wherein the processing the first variance image to obtain the first scratch image where merely the scratch is reserved comprises:
performing median filtration on the first variance image through the median filter in a vertical distance and the median filter in a horizontal direction, to obtain a first vertically-filtered image and a first horizontally-filtered image;
when the scratch in the to-be-processed video frame is a vertical scratch, subtracting the first horizontally-filtered image from the first vertically-filtered image to obtain the first scratch image; and
when the scratch in the to-be-processed video frame is the horizontal scratch, subtracting the first vertically-filtered image from the first horizontally-filtered image to obtain the first scratch image,

the processing the second variance image to obtain the second scratch image where merely the scratch is reserved comprises:
performing median filtration on the second variance image through the median filter in the vertical distance and the median filter in the horizontal direction, to obtain a second vertically-filtered image and a second horizontally-filtered image;
when the scratch in the to-be-processed video frame is the vertical scratch, subtracting the second horizontally-filtered image from the second vertically-filtered image to obtain the second scratch image; and
when the scratch in the to-be-processed video frame is the horizontal scratch, subtracting the second vertically-filtered image from the second horizontally-filtered image to obtain the second scratch image,

wherein the scratch-repaired image is calculated through:
I deline =I−L white +L black ,

where I deline represents the scratch-repaire

Assignees

• Boe Technology Group Co Ltd

Inventors

• Duan Ran
• Zhu Dan
• Wu Yanhong
• Chen Guannan

Classifications

• G06T5/60

  using machine learning, e.g. neural networks · CPC title

• G06T5/77 Primary

  Retouching; Inpainting; Scratch removal · CPC title

• G06T5/70 Primary

  Denoising; Smoothing · CPC title

• G06T2207/20224

  Image subtraction · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title