Equatorial stitching of hemispherical images in a spherical image capture system

Patent metadata

Field	Value
Publication number	US-11195253-B2
Application number	US-202016871762-A
Country	US
Kind code	B2
Filing date	May 11, 2020
Priority date	Aug 12, 2015
Publication date	Dec 7, 2021
Grant date	Dec 7, 2021

Abstract

Hyper-hemispherical images may be combined to generate a rectangular projection of a spherical image having an equatorial stitch line along of a line of lowest distortion in the two images. First and second circular images are received representing respective hyper-hemispherical fields of view. A video processing device may project each circular image to a respective rectangular image by mapping an outer edge of the circular image to a first edge of the rectangular image and mapping a center point of the circular image to a second edge of the first rectangular image. The rectangular images may be stitched together along the edges corresponding to the outer edge of the original circular image.

First claim

The invention claimed is:

1. An imaging apparatus configured to generate a planar projection of a spherical image, the imaging apparatus comprising:
a processing apparatus;
a first camera facing a first direction;
a second camera facing a second direction different from the first direction; and
a non-transitory computer-readable apparatus comprising a storage medium, the storage medium comprising computer instructions, the computer instructions configured to, when executed by the processing apparatus, enable the imaging apparatus to:
obtain a first non-planar image via the first camera;
obtain a second non-planar image via the second camera;
map an outer portion associated with the first non-planar image to a bottom portion associated with a first planar image;
map an outer portion associated with the second non-planar image to a top portion associated with a second planar image;
generate the planar projection of the spherical image based at least on a combination of the bottom portion associated with the first planar image with the top portion associated with the second planar image;
divide the planar projection of the spherical image into a first portion associated with a right half of the first camera and a second portion associated with a left half of the first camera; and
rotate and combine the first portion and the second portion such that the right half and the left half form a centered view of the spherical image.

2. The imaging apparatus of claim 1 , wherein the computer instructions are further configured to, when executed by the processing apparatus, enable the imaging apparatus to:
map a center portion associated with the first non-planar image to a top portion associated with the first planar image; and
map a center portion associated with the second non-planar image to a bottom portion associated with the second planar image.

3. The imaging apparatus of claim 2 , wherein the top portion associated with the first planar image and the bottom portion associated with the second planar image each comprises a single pixel.

4. The imaging apparatus of claim 2 , wherein the computer instructions are further configured to, when executed by the processing apparatus, enable the imaging apparatus to:
map one or more inner portions associated with the first non-planar image to one or more rows associated with the first planar image, the one or more inner portions being disposed between the outer and the center portions associated with the first non-planar image, the one or more rows being disposed between the bottom and the top portions associated with the first planar image; and
map another one or more inner portions associated with the second non-planar image to another one or more rows associated with the second planar image, the another one or more inner portions being disposed between the outer and the center portions associated with the second non-planar image, the another one or more rows being disposed between the bottom and the top portions associated with the second planar image.

5. The imaging apparatus of claim 1 , wherein the generation of the planar projection of the spherical image comprises generating a two-dimensional projection associated with a rectangular projection.

6. The imaging apparatus of claim 1 , wherein:
the first camera is configured to capture a first hyper-hemispherical field of view;
the second camera is configured to capture a second hyper-hemispherical field of view; and
the first and second cameras are configured such that first and second hyper-hemispherical fields of view overlap in at least one region by a prescribed degree.

7. The imaging apparatus of claim 1 , wherein the combination comprises (i) alignment of the bottom portion associated with the first planar image with respect to the top portion associated with the second planar image, and (ii) performance of a stitching algorithm on the bottom portion associated with the first planar image and the top portion associated with the second planar image.

8. A method of performing equatorial stitching, comprising:
capturing a first hemispherical video via a first camera of a spherical capture device and a second hemispherical video via a second camera of the spherical capture device, where the first camera and the second camera are characterized by an equatorial region; and
for each frame of the first hemispherical video and the second hemispherical video:
mapping a first hemispherical image to a first rectangular image and a second hemispherical image to a second rectangular image within a processor of the spherical capture device;
stitching the first rectangular image and the second rectangular image into a stitched rectangular image using a stitching algorithm designed for rectangular images, where stitching occurs at the equatorial region;
dividing the stitched rectangular image into a first portion associated with a right half of the first camera and a second portion associated with a left half of the first camera;
rotating and combining the first portion and the second portion such that the right half and the left half form a re-oriented stitched rectangular image comprising a centered view of a spherical image; and

generating a rectangular video from a plurality of re-oriented stitched rectangular images.

9. The method of claim 8 , wherein capturing the first hemispherical video and the second hemispherical video comprises capturing hyper-hemispherical videos that overlap at the equatorial region.

10. The method of claim 9 , further comprising:
mapping the equatorial region of the first hemispherical image to a first bottom edge of the first rectangular image;
mapping a first center point of the first hemispherical image to a first top edge of the first rectangular image;
mapping the equatorial region of the second hemispherical image to a second top edge of the second rectangular image; and
mapping a second center point of the second hemispherical image to a second bottom edge of the second rectangular image.

11. The method of claim 8 , where the stitched rectangular image is further re-oriented by:
combining the first portion and the second portion such that the right half and the left half are contiguous.

12. The method of claim 11 , where the rectangular video is generated according to a desired field of view identified by user control inputs.

13. The method of claim 12 , further comprising transmitting the rectangular video to another device for display.

14. A method for generating a planar projection of a spherical image, the method comprising:
obtaining a first non-planar image via a first camera;
obtaining a second non-planar image via a second camera;
mapping a first longitudinal portion associated with the first non-planar image to a first equatorial portion associated with a first planar image;
mapping a second longitudinal portion associated with the second non-planar image to a second equatorial portion associated with a second planar image;
stitching the first equatorial portion of the first planar image to the second equatorial portion of the second planar image to generate the planar projection;
divide the planar projection into a right half and a left half; and
rotate and combine the right half and the left half to form a centered view of the planar projection.

15. The method of claim 14 , where the first non-planar image is obtained from a first non-planar video and the second non-planar image is obtained from a second non-planar video.

16. The method of claim 14 , further comprising:
mapping a first pixel to a top portion of the

Assignees

• Gopro Inc

Inventors

• Steel Joseph
• Macmillan Timothy

Classifications

• G06T3/0018 Primary

  Physics · mapped topic

• G06T3/0043

  Physics · mapped topic

• G06T3/047 Primary

  Fisheye or wide-angle transformations · CPC title

• G06T3/073

  Transforming surfaces of revolution to planar images, e.g. cylindrical surfaces to planar images · CPC title