System and method for perspective preserving stitching and summarizing views

Patent metadata

Field	Value
Publication number	US-9569874-B2
Application number	US-201514732345-A
Country	US
Kind code	B2
Filing date	Jun 5, 2015
Priority date	Jun 5, 2015
Publication date	Feb 14, 2017
Grant date	Feb 14, 2017

Abstract

A method and a system of stitching a plurality of views of a scene, the method including extracting points of interest in each view to include a point set from each of the plurality of image views of the scene, matching the points of interest and reducing an outlier, grouping the matched points of interest in a plurality of groups, determining a similarity transformation with a smallest rotation angle for each grouping of the match points, generating virtual matching points on a non-overlapping area of the plurality of image views, generating virtual matching points on an overlapping area for each of the plurality of image views, and calculating piecewise projective transformations for the plurality of image views.

First claim

What is claimed is:

1. A method of stitching a plurality of image views of a scene, the method comprising:
extracting points of interest in each view to comprise a point set from each of the plurality of image views of the scene;
matching the points of interest and reducing outliers;
grouping the matched points of interest in a plurality of groups;
determining a similarity transformation with smallest rotation angle for each grouping of the matched points;
generating virtual matching points on non-overlapping area of the plurality of image views;
generating virtual matching points on overlapping area for each of the plurality of image views; and
calculating piecewise projective transformations for the plurality of image views.

2. The method according to claim 1 , wherein the points of interest representations comprise translation-invariant representations of edge orientations.

3. The method according to claim 2 , wherein the points of interest representations comprise scale invariant feature transform (SIFT) points.

4. The method according to claim 1 is stored in a non-transitory computer-readable medium and executed by a processor.

5. The method according to claim 1 , wherein the plurality of views of a scene are remotely captured from an aerial view.

6. The method according to claim 1 , wherein each group of the plurality of matched points is used to calculate an individual similarity transformation, and then the rotation angles corresponding to the transformations are examined and a one with the smallest rotation angle is selected.

7. The method according to claim 1 , further comprising, when extracting the points of interest, for each of the plurality of views, finding key points and texture descriptors.

8. A system of stitching a plurality of views of a scene, the system comprising:
a non-transitory computer-readable medium storing data including the plurality of views of the scene; and
a processor processing the data on the non-transitory computer-readable medium,
wherein the processor extracts points of interest in each view stored on the computer-readable medium to comprise a point set from each of the plurality of image views of the scene,
wherein the processor matches the points of interest and reduces an outlier,
wherein the processor groups the matched points of interest in a plurality of groups,
wherein the processor determines a similarity transformation for each group of the matched points, and
wherein the processor calculates piecewise projective transformations for the plurality of image views.

9. The system according to claim 8 , wherein the processor calculates piecewise projective transformations for the plurality of image views on overlapping areas.

10. The system according to claim 8 , wherein the processor determines the similarity transformation with a smallest rotation angle for each group of the matched points.

11. The system according to claim 8 , wherein the processor calculates linearized transformations for the plurality of image views.

12. The system according to claim 8 , wherein the processor uses weighted linearized transformations to extrapolate non-overlapping areas.

13. The system according to claim 8 , wherein the processor generates virtual matching points on a non-overlapping area of the plurality of image views, and
wherein the processor generates virtual matching points on an overlapping area for each of the plurality of image views.

14. The system according to claim 8 , wherein the points of interest representations comprise translational invariant representations of edge orientations.

15. The system according to claim 14 , wherein the points of interest representations comprise scale invariant feature transform (SIFT) points.

16. The system according to claim 8 , wherein the plurality of views of a scene are remotely captured from an aerial view and stored on the non-transitory computer-readable medium for execution by the processor.

17. The system according to claim 8 , wherein each group of the plurality of matched points is used to calculate an individual similarity transformation, then the rotation angles corresponding to the transformations are examined and a one with the smallest rotation angle is selected by the processor.

18. A method of developing summary visualization of the visual content in a plurality of videos of a scene, the method comprising:
processing the videos to extract objects and activities of interest;
establishing a frame of reference with respect to a visual content of a video so that the objects in a scene provide a least distracting view of the scene according to a predetermined criteria;
for each candidate frame in the video, relating a portion of its content with a portion of another frame of a gallery frame by finding a common portion of visual content in the corresponding frames;
relating the portion of the candidate frame to a common frame of reference by finding a chain of successive relationships that relate a candidate frame content to the reference video frame;
visualizing an overall content by rendering contents of the video frames in common frame of reference as a mosaic; and
overlaying the objects and activities of interest on the mosaic.

19. The method as claimed in claim 18 , wherein a user interface provides interaction with overlays to see details of the objects and the activities including the original video.

20. The method as claimed in 18 , wherein the activities are tracks,
wherein the common reference is related to a geographic map, and
wherein a user interface queries and searches the objects and activities.

21. A system comprising a non-transitory computer-readable medium and a processor executing the method according to claim 18 .

22. A method of stitching a plurality of image views of a scene, the method comprising: extracting a set of matched points of interest from each pair of the plurality of image views of the scene;
grouping the matched points of interest in a plurality of groups;
determining a similarity transformation with a smallest rotation angle for each group of the matched points;
generating virtual matching points on a non-overlapping area of the plurality of image views;
generating virtual matching points on an overlapping area for each of the plurality of image views; and
calculating piecewise projective transformations for the plurality of image views.

Assignees

• IBM

Inventors

• Lin Chung-Ching
• Pankanti Sharathchandra U
• Ramamurthy Karthikeyan Natesan
• Aravkin Aleksandr Y
• Smith John R

Classifications

• G06V10/462 Primary

  Salient features, e.g. scale invariant feature transforms [SIFT] · CPC title

• G06T11/60 Primary

  Creating or editing images; Combining images with text · CPC title

• G06T3/4038

  Image mosaicing, e.g. composing plane images from plane sub-images · CPC title

• G06T2207/10032

  Satellite or aerial image; Remote sensing · CPC title

• G06T2207/20104

  Interactive definition of region of interest [ROI] · CPC title