Method and arrangement for improved image matching

Patent metadata

Field	Value
Publication number	US-9589369-B2
Application number	US-201013509760-A
Country	US
Kind code	B2
Filing date	Dec 7, 2010
Priority date	Dec 11, 2009
Publication date	Mar 7, 2017
Grant date	Mar 7, 2017

Abstract

A method and device for determining at least one set of matching attributes between a plurality of images is disclosed. Two dimensional images are projected to a three dimensional space then searched for matching attributes. Searching for corresponding attributes is much easier and computationally less intensive in the 3D space compared to a search in the 2D space. The method includes steps of projecting at least part of the images of said plurality to a 3-dimensional space resulting in a plurality of 3-D projected images (image 1 _3D, image 2 _3D), searching for at least one corresponding set of elements within the 3D projected images of the plurality of 3-D projected images, and calculating back said corresponding elements within the original images of the plurality and providing said corresponding elements within said original images as said at least one set of matched attributes.

First claim

The invention claimed is:

1. An image processing method for determining at least one set of matched attributes (pixelli, pixel 2 j ; objectli, object 2 j ) between a plurality of images (image 1 , image 2 ), said method comprising the steps of:
using a processor and a memory to create a spatial 3-dimensional search space by projecting at least part of the images of said plurality of images to said spatial 3-dimensional space resulting in a plurality of spatial 3-dimensional projected images(imagel_3D, image 2 _3D);
using the processor and the memory to search for at least one set of corresponding spatial 3-dimensional elements within the spatial 3-dimensional projected images of the plurality of spatial 3-dimensional projected images; and
using the processor and the memory to calculate back said corresponding spatial 3-dimensional elements of said at least one set to corresponding 2-dimensional attributes within the plurality of images and providing said corresponding 2-dimensional attributes (pixelli, pixel 2 j ; objectli, object 2 j ) within said images as said at least one set of matched attributes.

2. A method according to claim 1 further comprising the steps of:
selecting one image (image 1 ) of said plurality as a reference image;
identifying a preliminary attribute on said reference image; and
identifying the spatial 3-dimensional projection of said preliminary attribute as a spatial 3-dimensional element (pixel3D, object3D) of said at least one set of corresponding spatial 3-dimensional elements whereby at least one corresponding spatial 3-dimensional element is searched in the spatial 3-dimensional projected images of the other images of the plurality of images.

3. A method according to claim 2 wherein said preliminary attribute comprises a pixel and wherein said at least one corresponding spatial 3-dimensional element is searched by forming a volume body around the spatial 3-dimensional projection of said pixel such that projected pixels of the spatial 3-dimensional projected images of the other images of the plurality which are falling within the contours of said volume body are identified as corresponding spatial 3-dimensional elements for said spatial 3-dimensional projection of said pixel.

4. A method according to claim 3 wherein said volume body is an ellipsoid centered around said spatial 3-dimensional projection of said pixel.

5. A method according to claim 1 wherein an element of said at least one set of corresponding spatial 3-dimensional elements comprises at least one projected spatial 3-dimensional object obtained from one or more spatial 3-dimensional projections of respective reference images selected from said plurality; and
wherein a corresponding spatial 3-dimensional element is determined as a group of spatial 3-dimensional projections of pixels from another projected image of said plurality of images which spatial 3-dimensional projections of pixels are falling within the contours of a volume body approximating said at least one projected spatial 3-dimensional object, such that said at least one set of matched attributes is obtained by the back calculation of said at least one projected spatial 3-dimensional object and of said group to the original images in the 2-dimensional domain.

6. A method according to claim 5 further comprising a step of 2-dimensional image recognition on said respective reference images such that to said at least one projected spatial 3-dimensional object at least one recognized object on said respective reference images is associated.

7. A method according to claim 5 further comprising a step of spatial 3-dimensional image recognition on said one or more spatial 3D projections of said respective reference images, such that to said at least one projected spatial 3-dimensional object at least one recognized object resulting from said spatial 3-dimensional image recognition is associated.

8. A method according to claim 5 wherein said volume body is selected from a set of volume models associated to said at least one projected spatial 3-dimensional object.

9. A method according to claim 5 further comprising a step of adapting said volume body by fitting said volume body with said at least one projected spatial 3-dimensional object.

10. A method according to claim 1 such that the matched attributes of said at least one set of matched attributes, and pertaining to said image, will be associated to said at least one recognized object on said respective reference images.

11. A method according to claim 1 , for determining a motion vector between two images (image 21 ( t− 1), image 22 ( t )) of a video sequence (video 2 ) taken at different instances in time (t−1, t), said method further comprising the steps of:
determining a matched pixel (p 21 ) within a first (image 21 ( t− 1)) of said two images at the earliest instance in time (t−1), for a start pixel (p 11 ) of a 2-dimensional motion vector determined between two images (image 11 ( t −1), image 12 ( t )) taken at said different instances in time (t−1, t) of a first video sequence (video 1 ); and
projecting said 2-dimensional motion vector of said first video sequence to the spatial 3-dimensional space, for obtaining a spatial 3-dimensional motion vector (mv 1 ), wherein said spatial 3-dimensional motion vector (mv 1 ) is applied to the spatial 3-dimensional projection (p 21 _3D(t−1)) of said matched pixel (p 21 ) to thereby obtain a corresponding end point (p 22 _3D(t−1)) which is further back projected to an end pixel (p 22 ) in a second image (image 22 ( t )) of said two images such that said matched pixel (p 21 ) and said end pixel (p 22 ) define said motion vector between said two images of said video sequence.

12. A method according to claim 1 , for determining a motion vector between two images (image 21 ( t− 1), image 22 ( t )) of a video sequence (video 2 ) taken at different instances in time (t−1, t), further comprising the step of:
determining respective matched pixels (p 21 , p 22 ) within said two images for the start (p 11 ) and end (p 12 ) pixels of a 2-dimensional motion vector determined between two images (image 11 ( t −1), image 12 ( t )) taken at said different instances in time (t−1, t) of a first video sequence (video 1 ), said pair of respective matched pixels (p 21 , p 22 ) in said two images (image 21 ( t− 1), image 22 ( t )) thereby defining start and end pixels of said motion vector of said video sequence.

13. An image processing device for determining at least one set of matched attributes between a plurality of images, said device comprising a processor and a memory configured to perform the following steps:
creating a spatial 3-dimensional search space by projecting at least part of the images of said plurality of images to said spatial 3-dimensional space resulting in a plurality of spatial 3-dimensional projected images;
searching for at least one set of corresponding spatial 3-dimensional elements within the spatial 3-dimensional projected images of the plurality of spatial 3-dimensional projected images; and
calculating back said corresponding spatial 3-dimensional elements of said at least one set to corresponding 2-dimensional attributes within the plurality of images and providing said corresponding 2-dimensional attributes within said images as said at least one set of matched attributes.

14. A device according to claim 13 wherein the processor and the memory are further configured to perform the steps of:
selecting one image (image 1 ) of said plurality as a reference image;
identifying a preliminary attribute on said reference image; and
identifying the spatial 3-dimensi

Assignees

• Tytgat Donny
• Macq Jean-Francois
• Lievens Sammy
• Aerts Maarten
• Alcatel Lucent

Inventors

• Tytgat Donny
• Macq Jean-Francois
• Lievens Sammy
• Aerts Maarten

Classifications

• G06T7/0071

  Physics · mapped topic

• G06T2207/10021

  Stereoscopic video; Stereoscopic image sequence · CPC title

• G06T7/0075

  Physics · mapped topic

• G06K9/00201

  Physics · mapped topic

• G06T7/2086 Primary

  Physics · mapped topic