Rectification techniques for heterogeneous camera arrays

Patent metadata

Field	Value
Publication number	US-9501826-B2
Application number	US-201414270687-A
Country	US
Kind code	B2
Filing date	May 6, 2014
Priority date	May 6, 2014
Publication date	Nov 22, 2016
Grant date	Nov 22, 2016

Abstract

Rectification techniques for camera arrays in which the resolutions, fields of view, and/or pixel sizes of various cameras may differ from one another are described. In one embodiment, for example, an apparatus may comprise logic, at least a portion of which is in hardware, the logic to receive a captured image array captured by a heterogeneous camera array, select a rectification process for application to the captured image array, identify a set of rectification maps for the selected rectification process, and apply the identified set of rectification maps to the captured image array to obtain a rectified image array. Other embodiments are described and claimed.

First claim

What is claimed is:

1. An apparatus, comprising:
logic, at least a portion of which is in hardware, the logic to:
identify respective rotation matrices for each of a plurality of cameras in a heterogeneous camera array and a composite rotation matrix for the heterogeneous camera array;
receive a captured image array generated by the heterogeneous camera array;
select a rectification process for application to the captured image array;
identify a set of rectification maps for the selected rectification process, the set of rectification maps generated based on the respective rotation matrices for the plurality of cameras in the heterogeneous camera array and on the composite rotation matrix for the heterogeneous camera array; and
apply the identified set of rectification maps to the captured image array to obtain a rectified image array.

2. The apparatus of claim 1 , the logic to generate the identified set of rectification maps in response to the selection of the rectification process.

3. The apparatus of claim 1 , the logic to perform a calibration process, the calibration process to comprise determining the respective rotation matrices for each of the plurality of cameras in the heterogeneous camera array, the composite rotation matrix for the heterogeneous camera array, and respective intrinsic parameter matrices for each of the plurality of cameras in the heterogeneous camera array.

4. The apparatus of claim 3 , the selected rectification process comprising a native rectification process according to which resolutions and fields-of-view for images in the rectified image array match resolutions and fields-of-view for corresponding images in the captured image array.

5. The apparatus of claim 4 , the set of rectification maps generated based on the respective rotation matrices and intrinsic parameter matrices for the plurality of cameras in the heterogeneous camera array and on the composite rotation matrix for the heterogeneous camera array.

6. The apparatus of claim 3 , the selected rectification process comprising a normalized rectification process according to which images in the rectified image array comprise a uniform resolution and a uniform field-of-view.

7. The apparatus of claim 6 , the set of rectification maps generated based on the respective rotation matrices and intrinsic parameter matrices for the plurality of cameras in the heterogeneous camera array, on the composite rotation matrix for the heterogeneous camera array, and on a composite intrinsic parameter matrix.

8. The apparatus of claim 7 , the calibration process comprising determining the composite intrinsic parameter matrix based on respective intrinsic parameter matrices for one or more cameras in the heterogeneous camera array that feature a target resolution and a target field of view for the normalized rectification process.

9. The apparatus of claim 1 , comprising:
a display;
a radio frequency (RF) transceiver; and
one or more RF antennas.

10. At least one non-transitory computer-readable storage medium comprising a set of instructions that, in response to being executed on a computing device, cause the computing device to:
identify respective rotation matrices for each of a plurality of cameras in a heterogeneous camera array and a composite rotation matrix for the heterogeneous camera array;
receive a captured image array generated by the heterogeneous camera array;
select a rectification process for application to the captured image array;
identify a set of rectification maps for the selected rectification process, the set of rectification maps generated based on the respective rotation matrices for the plurality of cameras in the heterogeneous camera array and on the composite rotation matrix for the heterogeneous camera array; and
apply the identified set of rectification maps to the captured image array to obtain a rectified image array.

11. The at least one non-transitory computer-readable storage medium of claim 10 , comprising instructions that, in response to being executed on the computing device, cause the computing device to generate the identified set of rectification maps in response to the selection of the rectification process.

12. The at least one non-transitory computer-readable storage medium of claim 10 , comprising instructions that, in response to being executed on the computing device, cause the computing device to perform a calibration process, the calibration process to comprise determining the respective rotation matrices for each of the plurality of cameras in the heterogeneous camera array, the composite rotation matrix for the heterogeneous camera array, and respective intrinsic parameter matrices for each of the plurality of cameras in the heterogeneous camera array.

13. The at least one non-transitory computer-readable storage medium of claim 12 , the selected rectification process comprising a native rectification process according to which resolutions and fields-of-view for images in the rectified image array match resolutions and fields-of-view for corresponding images in the captured image array.

14. The at least one non-transitory computer-readable storage medium of claim 13 , the set of rectification maps generated based on the respective rotation matrices and intrinsic parameter matrices for the plurality of cameras in the heterogeneous camera array and on the composite rotation matrix for the heterogeneous camera array.

15. The at least one non-transitory computer-readable storage medium of claim 12 , the selected rectification process comprising a normalized rectification process according to which images in the rectified image array comprise a uniform resolution and a uniform field-of-view.

16. The at least one non-transitory computer-readable storage medium of claim 15 , the set of rectification maps generated based on the respective rotation matrices and intrinsic parameter matrices for the plurality of cameras in the heterogeneous camera array, on the composite rotation matrix for the heterogeneous camera array, and on a composite intrinsic parameter matrix.

17. The at least one non-transitory computer-readable storage medium of claim 16 , the calibration process comprising determining the composite intrinsic parameter matrix based on respective intrinsic parameter matrices for one or more cameras in the heterogeneous camera array that feature a target resolution and a target field of view for the normalized rectification process.

18. A method, comprising:
identifying respective rotation matrices for each of a plurality of cameras in a heterogeneous camera array and a composite rotation matrix for the heterogeneous camera array;
receiving a captured image array generated by the heterogeneous camera array;
selecting, by a processor circuit, a rectification process for application to the captured image array;
identifying a set of rectification maps for the selected rectification process, the set of rectification maps generated based on the respective rotation matrices for the plurality of cameras in the heterogeneous camera array and on the composite rotation matrix for the heterogeneous camera array; and
applying the identified set of rectification maps to the captured image array to obtain a rectified image array.

19. The method of claim 18 , comprising generating the identified set of rectification maps in response to the selection of the rectification process.

20. The method of claim 18 , comprising performing a calibration process, the calibration process to comprise determini

Assignees

• Intel Corp

Inventors

• Seshadrinathan Kalpana
• Nestares Oscar

Classifications

• H04N5/2628 Primary

  Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation · CPC title

• H04N23/698

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• H04N25/60

  Noise processing, e.g. detecting, correcting, reducing or removing noise · CPC title

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• G06T2207/30208

  Marker matrix · CPC title