Method and apparatus for identifying local features

Patent metadata

Field	Value
Publication number	US-9613256-B2
Application number	US-201314069504-A
Country	US
Kind code	B2
Filing date	Nov 1, 2013
Priority date	Jun 12, 2013
Publication date	Apr 4, 2017
Grant date	Apr 4, 2017

Abstract

A method of local feature identification comprises retrieving data representing a lightfield, forming an epipolar volume (V) from the retrieved data, applying a transform to epipolar planes retrieved from the epipolar volume, so as to represent the epipolar planes in a new space, identifying a plurality of epipolar lines, and identifying local features (fp) based on at least some of the epipolar lines.

First claim

The invention claimed is:

1. A method of local feature identification and description, comprising the steps of:
retrieving data representing a lightfield, wherein the lightfield is represented by a plenoptic function which comprises at least three dimensions;
forming an epipolar volume (V) from the retrieved data;
applying a transform to epipolar planes retrieved from said epipolar volume, so as to represent said epipolar planes in a new space in which detection of epipolar lines is easier;
identifying a plurality of epipolar lines in the new space;
identifying local features (fp) by selecting a point along at least some of said epipolar lines.

2. The method of claim 1 , wherein said transform is a Radon transform.

3. The method of claim 1 , wherein said transform is a Hough transform.

4. The method of claim 1 , wherein said transform is based on the relation
λ y +λ ln λ x=μ

wherein x and y are coordinates in the epipolar plane, λ is the compressed slope and μ is the outcome of the transform.

5. The method of claim 4 , comprising a step of identifying a line and dividing all the corresponding λi by the largest one, λi is the compressed slope of epipolar line i.

6. The method of claim 4 , further comprising a step of computing a scale-invariant value (g(s)) as an integral over λ wherein λ is the compressed slope of an epipolar line.

7. The method of claim 1 , wherein said transform is a Mellin transform.

8. The method of claim 1 , comprising a step of applying a low-pass filter to data representing said epipolar plane.

9. The method of claim 1 , comprising a step of applying an edge detection algorithm to data representing said epipolar plane.

10. The method of claim 1 , comprising a step of identifying prospective feature points with a corner/feature method applied to an initial image corresponding to said lightfield, and only considering said lines in said new space that correspond to said prospective feature points as possible feature points.

11. The method of claim 1 , comprising a step of determining from said new space lines (l) in said epipolar plane.

12. The method of claim 11 , comprising selecting points in said new space with an intensity higher than a predefined threshold, and determining lines in said epipolar plane that correspond to said selected point.

13. The method of claim 11 , comprising a step of pruning the set of detected lines and only retaining the most stable ones, for example the lines longer than a threshold τ2.

14. The method of claim 1 , comprising a post-processing step applied after said transform step in order to ignore lines for which variance of intensity or color along the points is above some threshold, wherein the post-processing step comprises, comparing the variance of intensity or color along said points with a threshold, identify points which have variance of intensity or color which is below the threshold; omitting those point which are identified.

15. The method of claim 1 , comprising a step of retrieving one point along at least one selected epipolar line and this point to identify a local feature.

16. The method of claim 15 , wherein said local feature is described with a feature vector (fp).

17. The method of claim 16 , wherein said local feature is described with a three dimensional SIFT or BRIEF feature vector (fp).

18. The method of claim 15 , wherein said local feature is described with a two dimensional feature vector (fp) by considering only the image plane in which the feature point falls.

19. The method of claim 1 , further comprising computing a significance ratio for at least some prospective feature points, and omitting those feature points whose significance ration is lower than a threshold.

20. The method of claim 19 , wherein the significance ratio for a prospective feature point depends on the number and/or orientation or epipolar lines passing through a neighborhood of said prospective feature point.

21. The method of claim 1 , comprising a step of matching said local features (fp) with local features extracted from a 2D, 2D and half or 3D image.

22. The method of claim 1 , further comprising a steps of, detecting the dominant orientation of an image which corresponds to said light field, and rotating each of a plurality of images, each of which corresponds to a light field, so that each of the plurality of image has its orientation aligned to a reference orientation computed based on the detected dominant orientation.

23. The method of claim 1 , further comprising steps of, applying a tilting transform to an image which corresponds to said light field, wherein the step of applying a tilting transform to an image comprises scaling and translating the image in one dimension.

24. The method according to claim 23 wherein the tilting transform comprises a Fourier and/or a Mellin transforms.

25. A method according to claim 9 further comprising the step of applying a STILT transform to the data representing said epipolar plane after the edge detection algorithm has been applied.

26. An apparatus for identifying local features in lightfield representing data, comprising a lightfield data retrieving device for retrieving data representing a lightfield; and data processing means programmed for:
forming an epipolar volume (V) from the retrieved data;
applying a transform to epipolar planes retrieved from said epipolar volume, so as to represent said epipolar planes in a new space;
identifying a plurality of epipolar lines;
identifying local features (fp) based on at least some of said epipolar lines.

27. A non-transitory machine readable medium with program code being stored arranged for causing a processor to carry out a method of, local feature identification and description, when said processor executes said program code, said method of local feature identification and description, comprising the steps of:
retrieving data representing a lightfield;
forming an epipolar volume (V) from the retrieved data;
applying a transform to epipolar planes retrieved from said epipolar volume, so as to represent said epipolar planes in a new space;
identifying a plurality of epipolar lines;
identifying local features (fp) based on at least some of said epipolar lines.

28. A method of local feature identification and description, comprising the steps of:
retrieving data representing a lightfield, wherein the light field is represented by a plenoptic function which comprises at least three dimensions;
forming an epipolar volume (V) from the retrieved data;
applying a transform to epipolar planes retrieved from said epipolar volume, so as to represent said epipolar planes in a new space in which detection of epipolar lines is easier;
identifying a plurality of epipolar lines in the new space;
identifying local features (fp) by selecting a point along at least some of said epipolar tines,
wherein said transform comprises the relation
λ y +λ ln λ x=μ

wherein one side of the epipolar plane lies on a first axis and a second side of the epipolar plane lies on a second axis, and wherein x and y are coordinates in the first and second axes respectively and thus are coordinates in the epipolar plane, and wherein λ is the compressed slope and μ is the outcome of the transform; and wherein the method further comprises the step of,
compensating for scaling of the plenoptic images

Assignees

• Vidinoti Sa
• Ecole Polytechnique Fed De Lausanne (Epfl)

Inventors

• Ghasemi Alireza
• Rime Laurent
• Vetterli Martin

Classifications

• G06T2207/20212

  Image combination · CPC title

• G06T7/0095

  Physics · mapped topic

• G06T2207/10052

  Images from lightfield camera · CPC title

• G06T2207/20061

  Hough transform · CPC title

• G06K9/00201 Primary

  Physics · mapped topic