Cylindrical panorama hardware

Patent metadata

Field	Value
Publication number	US-12593134-B2
Application number	US-202318459586-A
Country	US
Kind code	B2
Filing date	Sep 1, 2023
Priority date	Dec 31, 2014
Publication date	Mar 31, 2026
Grant date	Mar 31, 2026

Abstract

A panoramic video camera comprises a plurality of image sensors which are configured to capture a plurality of frames at a time; an image processing circuitry configured to generate a frame read signal to read the plurality of frames generated by the plurality of camera sensors, apply a cylindrical mapping function to map the plurality of frames to a cylindrical image plane and stitch the cylindrically mapped plurality of frames together in the cylindrical image plane based on a plurality of projection parameters.

First claim

The invention claimed is:

1 . An image processing device, comprising:
a processor configured to:
read, in synchronization, a plurality of frames captured by a plurality of camera sensors;
calculate a plurality of alignment parameters and a plurality of projection parameters for the plurality of camera sensors, wherein the plurality of alignment parameters and the plurality of projection parameters are used in stitching images generated by the plurality of camera sensors;
apply a cylindrical mapping function to map the plurality of frames to a cylindrical image plane; and
stitch the cylindrically mapped plurality of frames together in the cylindrical image plane based on the calculated plurality of projection parameters and the calculated alignment parameters, wherein for a given target alignment parameter beginning stitching with a horizontal overlap having a factory-calibrated width that is initially determined based on a factory alignment parameter and is subsequently adjusted by increasing or decreasing the width of the horizontal overlap based on objects detected in both a first region overlapping with corresponding adjacent camera sensors and a second region non-overlapping with the corresponding adjacent camera sensors within the field of view of each camera sensor of the plurality of camera sensors.

2 . The image processing device according to claim 1 , wherein
for the given target alignment parameter, adjusting the horizontal overlap for each successive frame over the plurality of frames, the horizontal overlap is progressively increased or decreased to be nearer to the target alignment parameter until, for the final frame of the plurality of frames, the horizontal overlap becomes equal to the target alignment parameter.

3 . The image processing device according to claim 1 , further comprising the plurality of camera sensors configured to capture the plurality of frames.

4 . The image processing device according to claim 1 , wherein the processor is further configured to:
generate a plurality of frame read signals; and read, in synchronization, the plurality of frames based on the generated plurality of frame read signals.

5 . The image processing device according to claim 1 , wherein a projection parameter of the plurality of projection parameters for a corresponding camera sensor of the plurality of camera sensors is calculated to minimize parallax disparity in the horizontal overlap.

6 . The image processing device according to claim 1 , wherein the processor is further configured to dynamically adjust the horizontal overlap.

7 . The image processing device according to claim 6 , wherein the horizontal overlap is dynamically adjusted based on the objects in the field of view of the plurality of camera sensors.

8 . The image processing device according to claim 1 , wherein the plurality of projection parameters is determined based on factory calibration and runtime calibration.

9 . The image processing device according to claim 1 , wherein the processor is further configured to determine a stitch line between a left overlap region and a right overlap region for each of the plurality of frames.

10 . The image processing device according to claim 9 , wherein the stitch line is determined based on pixel differences between the left overlap region and the right overlap region.

11 . The image processing device according to claim 9 , wherein the processor is further configured to adjust the stitch line based on user input.

12 . The image processing device according to claim 9 , wherein the processor is further configured to blend the left overlap region and the right overlap region based on pixel differences between co-sited pixels in the left overlap region and the right overlap region.

13 . The image processing device according to claim 1 , wherein the processor is further configured to:
calculate color differences between adjacent camera sensors of the plurality of camera sensors; and control color and light response of each camera sensor of the plurality of camera sensors based on the calculated color differences.

14 . The image processing device according to claim 1 , wherein each of the plurality of frame read signals is initialized by a switch.

15 . The image processing device according to claim 14 , wherein the switch is controlled by a sensor control interface by broadcasting a control signal to the plurality of camera sensors.

16 . An image processing method, comprising:
reading, in synchronization, a plurality of frames captured by a plurality of camera sensors; calculating a plurality of alignment parameters and a plurality of projection parameters for the plurality of camera sensors, wherein the plurality of alignment parameters and the plurality of projection parameters are used in stitching images generated by the plurality of camera sensors; applying a cylindrical mapping function to map the plurality of frames to a cylindrical image plane; and stitching the cylindrically mapped plurality of frames together in the cylindrical image plane based on the calculated plurality of projection parameters and the calculated alignment parameters, wherein for a given target alignment parameter beginning stitching with a horizontal overlap having a factory-calibrated width that is initially determined based on a factory alignment parameter and is subsequently adjusted by increasing or decreasing the width of the horizontal overlap based on objects detected in both a first region overlapping with corresponding adjacent camera sensors and a second region non-overlapping with the corresponding adjacent camera sensors within the field of view of each camera sensor of the plurality of camera sensors.

17 . The image processing method according to claim 16 , wherein
for the given target alignment parameter, adjusting the horizontal overlap for each successive frame over the plurality of frames, the horizontal overlap is progressively increased or decreased to be nearer to the target alignment parameter until for the final frame of the plurality of frames, and the horizontal overlap is equal to the target alignment parameter.

18 . A non-transitory computer readable storage medium, having stored thereon, a set of computer-executable instructions that causes a computer to perform the steps comprising:
reading, in synchronization, a plurality of frames captured by a plurality of camera sensors; calculating a plurality of alignment parameters and a plurality of projection parameters for the plurality of camera sensors, wherein the plurality of alignment parameters and the plurality of projection parameters are used in stitching images generated by the plurality of camera sensors; applying a cylindrical mapping function to map the plurality of frames to a cylindrical image plane; and stitching the cylindrically mapped plurality of frames together in the cylindrical image plane based on the calculated plurality of projection parameters and the calculated alignment parameters, wherein for a given target alignment parameter beginning stitching with a horizontal overlap having a factory-calibrated width that is initially determined based on a factory alignment parameter and is subsequently adjusted by increasing or decreasing the width of the horizontal overlap based on objects detected in both a first region overlapping with corresponding adjacent camera sensors and a second region non-overlapping with the corresponding adjacent camera sensors within the field of view of each camera sensor of the plurality of c

Assignees

• Gn Audio As

Inventors

• Gupta Yashket

Classifications

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• G06T3/4038

  Image mosaicing, e.g. composing plane images from plane sub-images · CPC title

• G06T7/73

  using feature-based methods · CPC title

• H04N23/45 Primary

  for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images · CPC title

• G06F9/3004

  to perform operations on memory · CPC title