Image processing apparatus, imaging apparatus, and image processing method

Patent metadata

Field	Value
Publication number	US-9760999-B2
Application number	US-201414573040-A
Country	US
Kind code	B2
Filing date	Dec 17, 2014
Priority date	Jun 20, 2012
Publication date	Sep 12, 2017
Grant date	Sep 12, 2017

Abstract

An image processing apparatus includes an image acquisition unit that acquires a plurality of images, a corresponding point acquisition unit, a first fundamental matrix calculation unit, a depth calculation unit, a corresponding point extraction unit, and a fundamental matrix determination unit. The corresponding point acquisition unit acquires a plurality of first corresponding points of the images. The first fundamental matrix calculation unit calculates a first fundamental matrix based on the first corresponding points. The depth calculation unit calculates depths for the first corresponding points based on the first fundamental matrix. The corresponding point extraction unit extracts a plurality of second corresponding points from the first corresponding points based on the depths. The fundamental matrix determination unit calculates a second fundamental matrix based on the second corresponding points.

First claim

What is claimed is:

1. An image processing apparatus comprising:
one or more processor devices configured to:
acquire image data of a plurality of images;
acquire, from the image data of the plurality of images, a plurality of first corresponding points corresponding to each other between the plurality of images;
calculate a first fundamental matrix based on the plurality of first corresponding points;
calculate depths for the plurality of first corresponding points based on the first fundamental matrix;
extract a plurality of second corresponding points from the plurality of first corresponding points based on the depths such that the plurality of second corresponding points include a nearest one of the plurality of first corresponding points and a farthest one of the plurality of first corresponding points;
calculate a second fundamental matrix based on the plurality of second corresponding points; and
correct a shake present between the plurality of images based on the second fundamental matrix.

2. The image processing apparatus according to claim 1 ,
wherein the one or more processor devices are configured to successively extract the plurality of second corresponding points from the plurality of first corresponding points based on the depths in order from the nearest one of the plurality of first corresponding points, and in order from the farthest one of the plurality of first corresponding points.

3. The image processing apparatus according to claim 1 ,
wherein the one or more processor devices are configured to:
evaluate a reliability of the plurality of first corresponding points based on the first fundamental matrix; and
extract the plurality of second corresponding points based on the reliability.

4. The image processing apparatus according to claim 1 ,
wherein the plurality of images are of a same subject captured at different times.

5. The image processing apparatus according to claim 1 ,
wherein the one or more processor devices are configured to perform three-dimensional reconstruction of the plurality of images based on the second fundamental matrix.

6. The image processing apparatus according to claim 1 ,
wherein the one or more processor devices are configured to calculate the first fundamental matrix by an iterative operation performed by randomly selecting a predetermined number of corresponding points from the plurality of first corresponding points.

7. An imaging apparatus comprising:
an image sensor configured to capture a plurality of images of a subject at different times and to output image data of the plurality of images; and
one or more processor devices configured to:
acquire the image data of the plurality of images;
acquire, from the image data of the plurality of images, a plurality of first corresponding points corresponding to each other between the plurality of images;
calculate a first fundamental matrix based on the plurality of first corresponding points;
calculate depths for the plurality of first corresponding points based on the first fundamental matrix;
extract a a plurality of second corresponding points from the plurality of first corresponding points based on the depths such that the plurality of second corresponding points include a nearest one of the plurality of first corresponding points and a farthest one of the plurality of first corresponding points;
calculate a second fundamental matrix based on the plurality of second corresponding points; and
correct a shake present between the plurality of images based on the second fundamental matrix.

8. The imaging apparatus according to claim 7 ,
wherein the one or more processor devices are configured to successively extract the plurality of second corresponding points from the plurality of first corresponding points based on the depths in order from the nearest one of the plurality of first corresponding points, and in order from the farthest one of the plurality of first corresponding points.

9. The imaging apparatus according to claim 7 ,
wherein the one or more processor devices are configured to:
evaluate a reliability of the plurality of first corresponding points based on the first fundamental matrix; and
extract the plurality of second corresponding points based on the reliability.

10. The imaging apparatus according to claim 7 ,
wherein the one or more processor devices are configured to perform three-dimensional reconstruction of the plurality of images based on the second fundamental matrix.

11. The imaging apparatus according to claim 7 ,
wherein the one or more processor devices are configured to calculate the first fundamental matrix by an iterative operation performed by randomly selecting a predetermined number of corresponding points from the plurality of first corresponding points.

12. An image processing method comprising:
acquiring, by one or more processor devices, image data of a plurality of images;
acquiring, by the one or more processor devices, from the image data of the plurality of images, a plurality of first corresponding points corresponding to each other between the plurality of images;
calculating, by the one or more processor devices, a first fundamental matrix based on the plurality of first corresponding points;
calculating, by the one or more processor devices, depths for the plurality of first corresponding points based on the first fundamental matrix;
extracting, by the one or more processor devices, a plurality of second corresponding points from the plurality of first corresponding points based on the depths such that the plurality of second corresponding points include a nearest one of the plurality of first corresponding points and a farthest one of the plurality of first corresponding points;
calculating, by the one or more processor devices, a second fundamental matrix based on the plurality of second corresponding points; and
correcting, by the one or more processor devices, a shake present between the plurality of images based on the second fundamental matrix.

13. The image processing method according to claim 12 ,
wherein extracting the plurality of second corresponding points comprises successively extracting the plurality of second corresponding points from the plurality of first corresponding points based on the depths in order from the nearest one of the plurality of first corresponding points, and in order from the farthest one of the plurality of first corresponding points.

14. The image processing method according to claim 12 ,
wherein extracting the plurality of second corresponding points comprises:
evaluating a reliability of the plurality of first corresponding points based on the first fundamental matrix; and
extracting the plurality of second corresponding points based on the reliability.

15. The image processing method according to claim 12 ,
wherein the plurality of images are of a same subject captured at different times.

16. The image processing method according to claim 12 , further comprising:
performing, by the one or more processor devices, three-dimensional reconstruction of the plurality of images based on the second fundamental matrix.

17. The image processing method according to claim 12 ,
wherein calculating the first fundamental matrix comprises calculating the first fundamental matrix by an iterative operation performed by randomly selecting a predetermined number of corresponding points from the plurality of first corresponding points.

Assignees

• Olympus Corp

Inventors

• Sakata Seiichiro

Classifications

• G06T7/73 Primary

  using feature-based methods · CPC title

• H04N23/683

  performed by a processor, e.g. controlling the readout of an image memory · CPC title

• H04N23/6811

  based on the image signal · CPC title

• H04N23/80

  Camera processing pipelines; Components thereof · CPC title

• H04N5/23267

  Electricity · mapped topic