Methods and apparatus for multi-encoder processing of high resolution content

Patent metadata

Field	Value
Publication number	US-11790488-B2
Application number	US-202117332953-A
Country	US
Kind code	B2
Filing date	May 27, 2021
Priority date	Jun 6, 2017
Publication date	Oct 17, 2023
Grant date	Oct 17, 2023

Abstract

Methods and apparatus for multi-encoder processing of high resolution content. In one embodiment, the method includes capturing high resolution imaging content; splitting up the captured high resolution imaging content into respective portions; feeding the split up portions to respective imaging encoders; packing encoded content from the respective imaging encoders into an A/V container; and storing and/or transmitting the A/V container. In another embodiment, the method includes retrieving and/or receiving an A/V container; splitting up the retrieved and/or received A/V container into respective portions; feeding the split up portions to respective imaging decoders; stitching the decoded imaging portions into a common imaging portion; and storing and/or displaying at least a portion of the common imaging portion.

First claim

What is claimed:

1. A camera apparatus, comprising:
a first imaging sensor;
a second imaging sensor;
processor apparatus; and
a non-transitory computer-readable medium in data communication with the processor apparatus and comprising one or more instructions which are configured to, when executed by the processor apparatus, cause the camera apparatus to:
capture a first hemispheric image of a scene via the first imaging sensor;
capture a second hemispheric image of the scene via the second imaging sensor;
generate a spherical image of the scene by stitching at least a portion of the first hemispheric image and at least a portion of the second hemispheric image;
project the spherical image of the scene onto six facets of a cube:
generate non-hemispheric images of the scene by splitting the spherical image projected onto the six facets of the cube, wherein individual ones of the non-hemispheric images of the scene include a portion of the spherical image projected onto the six facets of the cube and are encoded separately, and wherein the encoded individual ones are packed into A/V containers; and
store the non-hemispheric images of the scene, wherein the non-hemispheric images of the scene are separately decodable and combinable to provide a spherical view of the scene.

2. The camera apparatus of claim 1 , wherein the individual ones of the non-hemispheric images of the scene are stored in separate A/V containers.

3. The camera apparatus of claim 1 , wherein the individual ones of the non-hemispheric images of the scene are stored in separate tracks of a media file.

4. The camera apparatus of claim 1 , wherein the individual ones of the non-hemispheric images of the scene do not overlap.

5. The camera apparatus of claim 1 , wherein at least two of the non-hemispheric images of the scene overlap.

6. The camera apparatus of claim 1 wherein the spherical image projected onto the six facets of the cube are split into two non-hemispheric images, the individual ones of the two non-hemispheric images including the spherical image projected onto three facets of the cube.

7. The camera apparatus of claim 1 wherein the spherical image projected onto the six facets of the cube are split into two non-hemispheric images, the individual ones of the two non-hemispheric images including the spherical image projected onto four facets of the cube.

8. The camera apparatus of claim 1 wherein the spherical image projected onto the six facets of the cube are split into three non-hemispheric images, the individual ones of the three non-hemispheric images including the spherical image projected onto two facets of the cube.

9. The camera apparatus of claim 1 wherein the spherical image projected onto the six facets of the cube are split such that the individual ones of the non-hemispheric images include the spherical image projected onto at least two continuous facets.

10. A method for processing images captured via a camera apparatus, comprising:
capturing a first hemispheric image of a scene via a first imaging sensor;
capturing a second hemispheric image of the scene via a second imaging sensor;
generating a spherical image of the scene by stitching at least a portion of the first hemispheric image and at least a portion of the second hemispheric image;
projecting the spherical image of the scene onto six facets of a cube;
generating non-hemispheric images of the scene by splitting the spherical image projected onto the six facets of the cube,
wherein individual ones of the non-hemispheric images of the scene include a portion of the spherical image projected onto the six facets of the cube and are encoded separately, and
wherein the encoded individual ones are packed into ΔN containers; and
storing the non-hemispheric images of the scene, wherein the non-hemispheric images of the scene are separately decodable and combinable to provide a spherical view of the scene.

11. The method of claim 10 , wherein individual ones of the non-hemispheric images of the scene are stored in separate A/V containers.

12. The method of claim 10 , wherein the individual ones of the non-hemispheric images of the scene are stored in separate tracks of a media file.

13. The method of claim 10 , wherein the individual ones of the non-hemispheric images of the scene do not overlap.

14. The method of claim 10 , wherein at least two of the non-hemispheric images of the scene overlap.

15. The method of claim 10 wherein the spherical image projected onto the six facets of the cube are split into two non-hemispheric images, the individual ones of the two non-hemispheric images including the spherical image projected onto three facets of the cube.

16. The method of claim 10 wherein the spherical image projected onto the six facets of the cube are split into two non-hemispheric images, the individual ones of the two non-hemispheric images including the spherical image projected onto four facets of the cube.

17. The method of claim 10 wherein the spherical image projected onto the six facets of the cube are split into three non-hemispheric images, the individual ones of the three non-hemispheric images including the spherical image projected onto two facets of the cube.

18. The method of claim 10 , wherein the spherical image projected onto the six facets of the cube are split such that the individual ones of the non-hemispheric images include the spherical image projected onto at least two continuous facets.

19. A camera apparatus, comprising:
a first imaging sensor;
a second imaging sensor;
processor apparatus; and a non-transitory computer-readable medium in data communication with the processor apparatus and comprising one or more instructions which are configured to, when executed by the processor apparatus, cause the camera apparatus to:
capture a first hemispheric image of a scene via the first imaging sensor;
capture a second hemispheric image of the scene via the second imaging sensor;
generate a spherical image of the scene by stitching at least a portion of the first hemispheric image and at least a portion of the second hemispheric image;
project the spherical image of the scene onto six facets of a cube;
generate non-hemispheric images of the scene by splitting the spherical image projected onto the six facets of the cube,
wherein individual ones of the non-hemispheric images of the scene include a portion of the spherical image projected onto the six facets of the cube and are encoded separately,
wherein the encoded individual ones are packed into A/V containers, and
further wherein the spherical image projected onto the six facets of the cube are split such that the individual ones of the non-hemispheric images include the spherical image projected onto at least two continuous facets; and
store the non-hemispheric images of the scene, wherein the non-hemispheric images of the scene are separately decodable and combinable to provide a spherical view of the scene.

20. The camera apparatus of claim 19 , wherein the spherical image projected onto the six facets of the cube are split into two non-hemispheric images, the individual ones of the two non-hemispheric images including the spherical image projected onto three or four facets of the cube.

Assignees

• Gopro Inc

Inventors

• Newman David
• Stimm Daryl
• Abbas Adeel

Classifications

• G06T3/4038 Primary

  Image mosaicing, e.g. composing plane images from plane sub-images · CPC title

• G06T3/0018

  Physics · mapped topic

• G06T3/0062

  Physics · mapped topic

• H04N19/80

  Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation (H04N19/635, H04N19/86 take precedence) · CPC title

• H04N21/21805 Primary

  enabling multiple viewpoints, e.g. using a plurality of cameras · CPC title