Image processing method and device and non-transitory computer-readable medium

Patent metadata

Field	Value
Publication number	US-10674086-B2
Application number	US-201815924428-A
Country	US
Kind code	B2
Filing date	Mar 19, 2018
Priority date	Jun 8, 2017
Publication date	Jun 2, 2020
Grant date	Jun 2, 2020

Abstract

Image processing method and device are disclosed. The image processing method is inclusive of a step of obtaining a target image; a step of performing motion estimation on the target image so as to attain motion paths; a step of determining, on the basis of the motion paths, whether there exists a fluctuation meeting a predetermined requirement in the target image; and a step of taking, if there exists a fluctuation meeting the predetermined requirement in the target image, the target image as a reference image. The reference image is utilized to carry out inter frame motion estimation with respect to the respective images acquired.

First claim

What is claimed is:

1. An image processing method comprising:
obtaining a target image;
performing motion estimation on the target image so as to attain motion paths;
determining, based on the motion paths, whether there exists a fluctuation meeting a predetermined requirement in the target image; and
taking, if there exists a fluctuation meeting the predetermined requirement in the target image, the target image as a reference image, the reference image being used to carry out inter frame motion estimation with respect to respective images acquired, wherein
the performing motion estimation on the target image so as to attain motion paths includes conducting inter frame motion estimation and real motion estimation in regard to the target image so as to attain an inter frame motion trajectory and a real motion trajectory, respectively, and
the determining, based on the motion paths, whether there exists a fluctuation meeting a predetermined requirement in the target image includes judging whether there is a fluctuation meeting the predetermined requirement in the inter frame motion trajectory and the real motion trajectory, respectively, wherein
the image processing method further comprising:
after the determining, based on the motion paths, whether there exists a fluctuation meeting a predetermined requirement in the target image, gaining, if there is a fluctuation meeting the predetermined requirement in the inter frame motion trajectory and the real motion trajectory, respectively, a first duration of the fluctuation in the inter frame motion trajectory and a second duration of the fluctuation in the real motion trajectory,
wherein, the taking the target image as a reference image includes
letting, if the first duration and the second duration are both greater than a predetermine threshold, the target image be a reference image.

2. The image processing method according to claim 1 , wherein, the determining, based on the motion paths, whether there exists a fluctuation meeting a predetermined requirement in the target image includes
judging whether there is a fluctuation meeting the predetermined requirement in the real motion trajectory.

3. The image processing method according to claim 2 , further comprising:
after the determining, based on the motion paths, whether there exists a fluctuation meeting a predetermined requirement in the target image, gaining, if there is a fluctuation meeting the predetermined requirement in the real motion trajectory, a duration of the fluctuation in the real motion trajectory,
wherein, the taking the target image as a reference image includes
letting, if the duration is greater than a predetermined threshold, the target image be a reference image.

4. The image processing method according to claim 1 , further comprising:
conducting vertical correction in regard to the reference image so as to procure a corrected reference image which is utilized to carry out inter frame motion estimation with respect to the respective image acquired;
performing a smoothing process on the motion paths; and
carrying out motion compensation based on the motion paths after smoothing.

5. The image processing method according to claim 4 , wherein, the conducting vertical correction in regard to the reference image includes
calculating an average level of motion within each sampling period in the real motion trajectory;
obtaining a three dimensional reference image corresponding to the reference image; and
rotating, based on the average levels of motion, the three dimensional reference image, so as to cause a two dimensional unfolded view corresponding to the rotated three dimensional reference image to be perpendicular to a line of sight of a relevant user.

6. The image processing method according to claim 4 , wherein, the performing a smoothing process on the motion paths includes
conducting, by means of median filtering, the smoothing process in regard to the motion paths.

7. The image processing method according to claim 4 , wherein, the carrying out motion compensation based on the motion paths after smoothing includes
projecting the target image onto a spherical coordinate system so as to attain a spherical image;
carrying out coordinate conversion with respect to pixel points in the spherical image so as to gain a spherical image after image stabilization; and
transforming the spherical image after image stabilization into a two dimensional image.

8. An image processing device comprising:
a memory; and
a processor coupled to the memory, wherein
the processor
obtains a target image,
performs motion estimation on the target image so as to attain motion paths,
determines, based on the motion paths, whether there exists a fluctuation meeting a predetermined requirement in the target image,
takes, if there exists a fluctuation meeting the predetermined requirement in the target image, the target image as a reference image, the reference image being utilized to carry out inter frame motion estimation with respect to respective images acquired, wherein
when performing motion estimation on the target image so as to attain motion paths, the processor conducts inter frame motion estimation and real motion estimation in regard to the target image so as to attain an inter frame motion trajectory and a real motion trajectory, respectively, and
when determining, based on the motion paths, whether there exists a fluctuation meeting a predetermined requirement in the target image, the processor judges whether there is a fluctuation meeting the predetermined requirement in the inter frame motion trajectory and the real motion trajectory, respectively, and wherein
the processor,
after the processor's determining, based on the motion paths, whether there exists a fluctuation meeting a predetermined requirement in the target image, gains, if there is a fluctuation meeting the predetermined requirement in the inter frame motion trajectory and the real motion trajectory, respectively, a first duration of the fluctuation in the inter frame motion trajectory and a second duration of the fluctuation in the real motion trajectory,
wherein, the processor's taking the target image as a reference image includes
letting, if the first duration and the second duration are both greater than a predetermine threshold, the target image be a reference image.

9. A non-transitory computer-readable medium having computer-executable instructions for execution by a processing system, wherein, the computer-executable instructions, when executed, cause the processing system to carry out the image processing method according to claim 1 .

Assignees

• Wang Yiwei
• Liang Lingyan
• Ricoh Co Ltd

Inventors

• Wang Yiwei
• Liang Lingyan

Classifications

• H04N5/23248

  Electricity · mapped topic

• H04N5/23254 Primary

  Electricity · mapped topic

• G06T2207/30241

  Trajectory · CPC title

• H04N5/23296

  Electricity · mapped topic

• H04N5/23258

  Electricity · mapped topic