Systems and methods for spatially selective video coding

Patent metadata

Field	Value
Publication number	US-12126809-B2
Application number	US-202318144574-A
Country	US
Kind code	B2
Filing date	May 8, 2023
Priority date	Jun 20, 2016
Publication date	Oct 22, 2024
Grant date	Oct 22, 2024

Abstract

Encoded content is accessed. The encoded content includes an encoded first centrally located tile corresponding to a first centrally located tile of a first image, an encoded first peripherally located tile of the first image, and an encoded second peripherally located tile of a second image. The encoded first peripherally located tile is decoded to obtain a decoded first peripherally located tile. The encoded second peripherally located tile is decoded to obtain a decoded second peripherally located tile. The decoded first peripherally located tile and the decoded second peripherally located tile are stitched to obtain a stitched image portion. The stitched image portion is encoded to obtain an encoded stitched image portion. An encoded stitched image of the first image and the second image is obtained by combining the encoded first centrally located tile, and the encoded stitched image portion.

First claim

What is claimed is:

1. A system, comprising:
a processing apparatus that includes a first processor configured to execute instructions to:
access encoded content, wherein the encoded content includes:
an encoded first centrally located tile corresponding to a first centrally located tile of a first image,
an encoded first peripherally located tile of the first image, and
an encoded second peripherally located tile of a second image;

decode the encoded first peripherally located tile to obtain a decoded first peripherally located tile;
decode the encoded second peripherally located tile to obtain a decoded second peripherally located tile;
stitch the decoded first peripherally located tile and the decoded second peripherally located tile to obtain a stitched image portion;
encode the stitched image portion to obtain an encoded stitched image portion; and
obtain an encoded stitched image of the first image and the second image by combining the encoded first centrally located tile, and the encoded stitched image portion.

2. The system of claim 1 , wherein the encoded content is obtained from an imaging device comprising a second processor configured to execute instructions to:
partition the first image into the first centrally located tile and first peripherally located tiles, wherein the first peripherally located tiles include the first peripherally located tile;
partition the second image into a second centrally located tile and second peripherally located tiles, wherein the second peripherally located tiles include the second peripherally located tile, and wherein the first peripherally located tile and the second peripherally located tile include a stitching boundary;
encode the first centrally located tile, to obtain the encoded first centrally located tile, without reference to any of the first peripherally located tiles; and
encode the second centrally located tile, to obtain an encoded second centrally located tile, without reference to any of the second peripherally located tiles.

3. The system of claim 2 , wherein the first centrally located tile and the second centrally located tile are encoded without reference to peripherally located tiles using motion-constrained tile configurations of a High Efficiency Video Coding (HEVC) codec.

4. The system of claim 2 , wherein the first centrally located tile is encoded without reference to the first peripherally located tiles by restricting motion vectors so that they do not cross a tile boundary.

5. The system of claim 1 , wherein the first processor is further configured to execute instructions to:
omit decoding the encoded first centrally located tile responsive to a determination not to decode the encoded first centrally located tile.

6. The system of claim 5 , wherein the determination not to decode the encoded first centrally located tile is made based one of an image map, a look up table, or a rulebook.

7. The system of claim 1 , wherein the instructions to combine the encoded first centrally located tile and the encoded stitched image portion comprise to:
copy pixel values of the encoded first centrally located tile into the encoded stitched image.

8. The system of claim 1 , the first processor is further configured to execute instructions to:
decode the encoded stitched image for display in a panoramic image.

9. A method, comprising:
partitioning images into one or more centrally located tiles and one or more peripherally located tiles, wherein the images are partitioned such that the peripherally located tiles contain a stitching boundary and the centrally located tiles do not contain any stitching boundaries; and
encoding the images to obtain encoded images, wherein the centrally located tiles are encoded without reference to the peripherally located tiles, and wherein the encoded images include respective encoded centrally located tiles and respective encoded peripherally located tiles.

10. The method of claim 9 , further comprising:
modifying a tiling operation that partitions the images based on a processing order that maximizes or minimizes processing effort.

11. The method of claim 9 , wherein the centrally located tiles are obtained by encoding corresponding pixels of the images as individual motion-constrained tiles.

12. The method of claim 9 , wherein the centrally located tiles are encoded without reference to the peripherally located tiles using a motion-constrained tile configurations of a High Efficiency Video Coding (HEVC) codec.

13. The method of claim 9 , wherein the centrally located tiles are encoded without reference to the peripherally located tiles by restricting motion vectors so that the motion vectors do not cross a tile boundary.

14. The method of claim 9 , further comprising:
stitching the images, to obtain a stitched image, by decoding the respective encoded peripherally located tiles and omitting decoding the respective encoded centrally located tiles based on one of an image map, a look up table, or a rulebook.

15. The method of claim 14 , wherein stitching the images, to obtain the stitched image, comprises:
decoding the respective encoded peripherally located tiles; and
stitching the decoded respective encoded peripherally located tiles.

16. The method of claim 14 , further comprising:
encoding the stitched image to obtain an encoded stitch image, wherein encoding the stitched image comprises:
copying pixel values of the respective encoded centrally located tiles into the encoded stitched image.

17. A non-transitory computer-readable storage medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations comprising to:
receive encoded content, wherein the encoded content includes:
an encoded centrally located tile corresponding to a centrally located tile of a first image,
encoded first peripherally located tiles of the first image, and
encoded second peripherally located tiles of a second image;

decode the encoded first peripherally located tiles and the encoded second peripherally located tiles, and omit decoding the encoded centrally located tile;
stitch at least one of the decoded first peripherally located tile and at least one of the decoded second peripherally located tile to obtain a stitched image portion;
encode the stitched image portion to obtain an encoded stitched image portion; and
obtain an encoded stitched image of the first image and the second image by combining the encoded centrally located tile and the encoded stitched image portion.

18. The non-transitory computer-readable storage medium of claim 17 , wherein the centrally located tile is encoded without reference to peripherally located tiles using a motion-constrained tile configurations of a High Efficiency Video Coding (HEVC) codec.

19. The non-transitory computer-readable storage medium of claim 17 , wherein the centrally located tile is encoded by restricting motion vectors so that the motion vectors do not cross a tile boundary.

20. The non-transitory computer-readable storage medium of claim 17 , wherein the encoded centrally located tile does not contain stitching boundaries.

Assignees

• Gopro Inc

Inventors

• Abbas Adeel
• Adsumilli Balineedu Chowdary
• Polonichko Vadim

Classifications

• H04N23/698

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• H04N23/45

  for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images · CPC title

• H04N19/597

  specially adapted for multi-view video sequence encoding · CPC title

• H04N19/107

  between spatial and temporal predictive coding, e.g. picture refresh · CPC title

• H04N13/00

  Stereoscopic video systems; Multi-view video systems; Details thereof · CPC title