Method of fast and robust camera location ordering

Patent metadata

Field	Value
Publication number	US-9791264-B2
Application number	US-201514614224-A
Country	US
Kind code	B2
Filing date	Feb 4, 2015
Priority date	Feb 4, 2015
Publication date	Oct 17, 2017
Grant date	Oct 17, 2017

Abstract

A method to estimate a set of camera locations, in clockwise or counter-clockwise order, according to the videos captured by these cameras is described herein. In some embodiments, the cameras are assumed to be fixed, with no or very mild tilting angles and no rolling angles (the horizon is horizontal in each camera image). The difference of orientation (rolling angle) between each neighboring (closest) camera pair is able to be up to 45 degrees. Each camera is assumed to have overlapped views with at least one other camera. Each camera has one right neighboring camera and one left neighboring camera, except the first and the last cameras which have only one neighboring camera at one side. The locations of the cameras then are able to be expressed as a unique list counter-clockwise. The input videos are assumed to be synchronized in time.

First claim

What is claimed is:

1. A method programmed in a non-transitory memory of a device comprising:
a. performing background extraction, wherein performing background extraction generates a background extraction output including a moving object region of a camera;
b. performing pairwise camera correspondence extraction, wherein performing pairwise camera correspondence extraction utilizes the background extraction output and an image as input;
c. identifying neighboring cameras;
d. determining neighboring camera positioning; and
e. performing camera list topology deduction.

2. The method of claim 1 further comprising preparing settings and input.

3. The method of claim 1 wherein performing background extraction includes background modeling and determining reliable moving objects.

4. The method of claim 1 wherein performing pairwise camera correspondence extraction includes keypoint detection with subsampling, keypoint descriptor extraction and pairwise correspondence.

5. The method of claim 1 wherein performing pairwise camera correspondence extraction outputs corresponding keypoints between a first camera and a second camera.

6. The method of claim 5 wherein identifying neighboring cameras utilizes the keypoints between the first camera and the second camera as input.

7. The method of claim 1 wherein identifying neighboring cameras outputs a binary neighboring camera map.

8. The method of claim 7 wherein determining neighboring camera positioning utilizes a moving object region of a camera, corresponding keypoints between a first camera and a second camera and the binary neighboring camera map as input.

9. The method of claim 8 wherein determining neighboring camera positioning outputs a neighboring camera relative direction map.

10. The method of claim 1 wherein topology deduction includes relational swapping with loop resolving, nearest neighbor refinement and post-relational swapping.

11. The method of claim 1 wherein topology deduction utilizes a neighboring camera relative direction map and corresponding keypoints between a first camera and a second camera as input.

12. The method of claim 1 wherein topology deduction outputs a camera list topology.

13. A system comprising:
a. a plurality of camera devices each configured for capturing video content; and
b. a computing device configured for:
i. receiving the video content;
ii. performing background extraction of the video content, wherein performing background extraction generates a background extraction output including a moving object region of a camera;
iii. performing pairwise camera correspondence extraction, wherein performing pairwise camera correspondence extraction utilizes the background extraction output and an image as input;
iv. identifying neighboring cameras of the plurality of camera devices;
v. determining neighboring camera positioning; and
vi. performing camera list topology deduction.

14. The system of claim 13 wherein the computing device is further configured for preparing settings and input.

15. The system of claim 13 wherein performing background extraction includes background modeling and determining reliable moving objects.

16. The system of claim 13 wherein performing pairwise camera correspondence extraction includes keypoint detection with subsampling, keypoint descriptor extraction and pairwise correspondence.

17. The system of claim 13 wherein performing pairwise camera correspondence extraction outputs corresponding keypoints between a first camera and a second camera.

18. The system of claim 17 wherein identifying neighboring cameras utilizes the keypoints between the first camera and the second camera as input.

19. The system of claim 13 wherein identifying neighboring cameras outputs a binary neighboring camera map.

20. The system of claim 19 wherein determining neighboring camera positioning utilizes a moving object region of a camera, corresponding keypoints between a first camera and a second camera and the binary neighboring camera map as input.

21. The system of claim 20 wherein determining neighboring camera positioning outputs a neighboring camera relative direction map.

22. The system of claim 13 wherein topology deduction includes relational swapping with loop resolving, nearest neighbor refinement and post-relational swapping.

23. The system of claim 13 wherein topology deduction utilizes a neighboring camera relative direction map and corresponding keypoints between a first camera and a second camera as input.

24. The system of claim 13 wherein topology deduction outputs a camera list topology.

25. A camera device comprising:
a. a lens;
b. a sensor configured for acquiring video content;
c. a non-transitory memory for storing an application, the application for:
i. receiving the video content;
ii. performing background extraction of the video content, wherein performing background extraction generates a background extraction output including a moving object region of the camera device;
iii. performing pairwise camera correspondence extraction, wherein performing pairwise camera correspondence extraction utilizes the background extraction output and an image as input;
iv. identifying neighboring cameras of the plurality of camera devices;
v. determining neighboring camera positioning; and
vi. performing camera list topology deduction; and

d. a processing component coupled to the memory, the processing component configured for processing the application.

26. The camera device of claim 25 wherein the application is further configured for preparing settings and input.

27. The camera device of claim 25 wherein performing background extraction includes background modeling and determining reliable moving objects.

28. The camera device of claim 25 wherein performing pairwise camera correspondence extraction includes keypoint detection with subsampling, keypoint descriptor extraction and pairwise correspondence.

29. The camera device of claim 25 wherein performing pairwise camera correspondence extraction outputs corresponding keypoints between a first camera and a second camera.

30. The camera device of claim 29 wherein identifying neighboring cameras utilizes the keypoints between the first camera and the second camera as input.

31. The camera device of claim 25 wherein identifying neighboring cameras outputs a binary neighboring camera map.

32. The camera device of claim 31 wherein determining neighboring camera positioning utilizes a moving object region of a camera, corresponding keypoints between a first camera and a second camera and the binary neighboring camera map as input.

33. The camera device of claim 32 wherein determining neighboring camera positioning outputs a neighboring camera relative direction map.

34. The camera device of claim 25 wherein topology deduction includes relational swapping with loop resolving, nearest neighbor refinement and post-relational swapping.

35. The camera device of claim 25 wherein topology deduction utilizes a neighboring camera relative direction map and corresponding keypoints between a first camera and a second camera as input.

36. The camera device of claim 25 wherein topology deduction outputs a c

Assignees

• Sony Corp

Inventors

• Liu Cheng-Yi
• Berestov Alexander

Classifications

• H04N23/681 Primary

  Motion detection · CPC title

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• G06T2207/20012

  Locally adaptive · CPC title

• G06T7/143

  involving probabilistic approaches, e.g. Markov random field [MRF] modelling · CPC title

• G06T2207/20072

  Graph-based image processing · CPC title