Image processing device, image processing method, and recording medium storing program

Patent metadata

Field	Value
Publication number	US-10645284-B2
Application number	US-201815967947-A
Country	US
Kind code	B2
Filing date	May 1, 2018
Priority date	May 26, 2017
Publication date	May 5, 2020
Grant date	May 5, 2020

Abstract

An image processing apparatus, an image processing method, and a recording medium storing a program for causing a computer to execute the image processing method. The image processing apparatus and the image processing method includes obtaining a captured image captured using a wide-angle lens, detecting movement of an image processing apparatus, converting the captured image into a wide-angle image that at least partially involves a curved plane, calculating an amount of correction according to the movement detected by the detecting, and correcting the captured image by the amount of correction.

First claim

What is claimed is:

1. An image processing apparatus, comprising:
circuitry configured to
obtain a captured image captured using a wide-angle lens,
detect translational movement of the image processing apparatus by determining a first three-dimensional vector, which represents displacement of the image processing apparatus from a first time to a second time,
convert the captured image into a wide-angle image that is at least partially defined on a curved plane, and
calculate an amount of correction based on the wide-angle image and the detected translational movement by calculating a second three-dimensional vector based on the determined first three-dimensional vector, and correct the captured image using the calculated second three-dimensional vector.

2. The image processing apparatus according to claim 1 , wherein the circuitry is further configured to convert the captured image into a sphere, and calculate an amount of movement of a point on the sphere by calculating the second three-dimensional vector, according to the determined first three-dimensional vector, and correct the captured image by the calculated amount of correction, which is based on the calculated second three-dimensional vector.

3. The image processing apparatus according to claim 2 , wherein the circuitry is further configured to
refer to association data in which planar coordinates of the captured image and spherical coordinates of the sphere are associated with each other, and convert first planar coordinates of the captured image into corresponding coordinates on the sphere,
change the coordinates of the sphere according to the detected translational movement, and
replace second planar coordinates that are associated with the changed coordinates of the sphere in the association data with the first planar coordinates.

4. The image processing apparatus according to claim 3 , wherein the circuitry is further configured to
perform inverse conversion of a conversion in perspective projection, in which a point in three-dimensional space is projected onto the point on the sphere, on a first point of a first sphere, to convert a point in three-dimensional space before the translational movement into a point in three-dimensional space after the translational movement according to x, y, z components of the detected translational movement,
perform the conversion in perspective projection for the point in three-dimensional space after the translational movement to calculate a second point on a second sphere, and
regard the second point on the second sphere as the coordinates of the sphere.

5. The image processing apparatus according to claim 4 , wherein the circuitry is further configured to calculate a difference between the first point of the first sphere and the second point of the second sphere, and add the difference to the first point of the first sphere to obtain a point on a spherical surface of the sphere.

6. The image processing apparatus according to claim 4 , wherein the circuitry is further configured to
calculate a difference between the first point of the first sphere and the second point of the second sphere,
add the difference to the first point of the first sphere to obtain a point as the changed coordinates of the sphere, and
replace the second planar coordinates associated with the changed coordinates of the sphere in the association data with the first planar coordinates.

7. The image processing apparatus according to claim 3 , wherein the circuitry is further configured to
refer to weighting data in which a weighting indicating a degree of correction is associated with the spherical coordinates, and
change coordinates of the sphere by the weighting associated with the spherical coordinates, according to the detected translational movement.

8. The image processing apparatus according to claim 7 ,
wherein the circuitry is further configured to determine the weighting data using a distance to an object estimated according to the spherical coordinates, and
wherein the spherical coordinates are associated with the weighting being greater when the distance is shorter and smaller when the distance is longer.

9. The image processing apparatus according to claim 7 ,
wherein the weighting data is specified in advance according to a capturing scene, and
wherein the circuitry is further configured to correct the captured image using the weighting data, which is selected by a user.

10. The image processing apparatus according to claim 7 , wherein when an image captured by the image processing apparatus is displayed upon being divided into a plurality of mesh-like sections, the circuitry is further configured to generate weighting data according to distance data input by a user for each one of the mesh-like sections.

11. A method of processing an image, the method comprising:
obtaining a captured image captured using a wide-angle lens;
detecting translational movement of an image processing apparatus by determining a first three-dimensional vector, which represents displacement of the image processing apparatus from a first time to a second time;
converting the captured image into a wide-angle image that is at least partially defined on a curved plane;
calculating an amount of correction based on the wide-angle image and the translational movement detected in the detecting step by calculating a second three-dimensional vector based on the determined first three-dimensional vector; and
correcting the captured image using the calculated second three-dimensional vector.

12. A non-transitory computer-readable recording medium storing a program that causes a computer to execute a method comprising:
obtaining a captured image captured using a wide-angle lens;
detecting translational movement of an image processing apparatus by determining a first three-dimensional vector, which represents displacement of the image processing apparatus from a first time to a second time;
converting the captured image into a wide-angle image that is at least partially defined on a curved plane;
calculating an amount of correction based on the wide-angle image and the translational movement detected in the detecting step by calculating a second three-dimensional vector based on the determined first three-dimensional vector; and
correcting the captured image using the calculated second three-dimensional vector.

Assignees

• Matsushita Kazufumi
• Ricoh Co Ltd

Inventors

• Matsushita Kazufumi

Classifications

• H04N13/282

  for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems · CPC title

• H04N5/23238 Primary

  Electricity · mapped topic

• H04N5/23267

  Electricity · mapped topic

• H04N5/23254

  Electricity · mapped topic

• H04N5/2251

  Electricity · mapped topic