Apparatus and methods for image encoding using spatially weighted encoding quality parameters

Patent metadata

Field	Value
Publication number	US-10965868-B2
Application number	US-201916363668-A
Country	US
Kind code	B2
Filing date	Mar 25, 2019
Priority date	Jun 17, 2016
Publication date	Mar 30, 2021
Grant date	Mar 30, 2021

Abstract

Methods and apparatus for encoding and decoding image data based on one or more parameters. In one embodiment, various spatial portions or regions of image data (e.g., a still or moving image) are weighted according to the perceived or measured quality. Processing for these weighted regions can be selectively altered or adjusted so as to optimize one or more operational parameters including for example processing and/or memory requirements, or speed.

First claim

What is claimed:

1. A computerized apparatus, comprising:
a non-transitory computer-readable apparatus having a storage medium, the storage medium comprising one or more computer-executable instructions that are configured to, when executed by one or more processor apparatus, cause the computerized apparatus to:
obtain visual content from an image capture device, the visual content comprising a plurality of spatial portions;
encode each of the plurality of spatial portions of the visual content using respective spatially weighted encoding quality parameters, the respective spatially weighted encoding quality parameters comprising information associated with resources configured for post-processing the visual content; and
generate one or more derivative content based on the encode of each of the plurality of spatial portions of the visual content.

2. The computerized apparatus of claim 1 , wherein the obtaining of the visual content from the image capture device comprises obtaining of the visual content generated from one or more optical sensors.

3. The computerized apparatus of claim 1 , wherein the obtaining of the visual content comprises generation of the visual content via performance of at least a render operation using at least a plurality of images, the render operation comprising a stitch of the at least plurality of images.

4. The computerized apparatus of claim 3 , wherein the stitch of the at least plurality of images comprises an evaluation of at least some of the respective spatially weighted encoding quality parameters, the evaluation enabling the computerized apparatus to weight image information contained in respective ones of the at least plurality of images based on the respective spatially weighted encoding quality parameters.

5. The computerized apparatus of claim 1 , wherein the one or more computer-executable instructions are further configured to, when executed by the one or more processor apparatus, to cause the computerized apparatus to decode one or more of the plurality of spatial portions of the visual content; and
wherein the generation of the one or more derivative content is further based on data relating to one or more capabilities associated with a display device that is configured to render the one or more derivative content after generation thereof.

6. The computerized apparatus of claim 1 , wherein the one or more computer-executable instructions are further configured to, when executed by the one or more processor apparatus, cause the computerized apparatus to allocate an amount of resources for a decode of the one or more of the plurality of spatial portions, the allocation based at least on respective one or more spatially weighted encoding quality parameters associated with the one or more spatial portions.

7. The computerized apparatus of claim 1 , wherein the respective spatially weighted encoding quality parameters are associated with a distribution curve that is descriptive of a quality level as a function of location within the visual content.

8. A non-transitory computer-readable apparatus comprising a storage medium having at least one computer program therein, the at least one computer program comprising a plurality of instructions configured to cause, when executed by a processor apparatus:
generation of content based on one or more images captured via at least one image capturing device;
based on at least one respective encoding parameter associated with each of a plurality of portions of the generated content, processing of each of the plurality of portions of the generated content; and
rendering of encoded content based at least on the processing of each of the plurality of portions of the generated content, portions of the encoded content being associated with respective levels of image quality corresponding to the respective encoding parameters.

9. The non-transitory computer-readable apparatus of claim 8 , wherein the plurality of instructions are further configured to cause, when executed by the processor apparatus:
pre-processing of the each of the plurality of portions of the generated content for storage;
storage of at least the pre-processed plurality of portions of the generated content along with the respective encoding parameters within a data structure; and
retrieval of the pre-processed plurality of portions of the generated content and the respective encoding parameters for said processing of each of the plurality of portions of the generated content.

10. The non-transitory computer-readable apparatus of claim 8 , wherein the generated content comprises an at least hemipherical image, and a center portion of the at least hemispherical image is associated with greater image quality as compared to one or more peripheral portions of the at least hemispherical image.

11. The non-transitory computer-readable apparatus of claim 8 , wherein the respective encoding parameters are associated with a mathematical distribution function that is descriptive of a quality level as a function of position within the generated content.

12. The non-transitory computer-readable apparatus of claim 11 , wherein the mathematical distribution function is based at least in part on a spatial distortion introduced by the image capturing device.

13. The non-transitory computer-readable apparatus of claim 8 , wherein the plurality of instructions are further configured to cause, when executed by the processor apparatus:
determination of the respective encoding parameters based at least in part on one or more parameters associated with at least one lens of the image capturing device.

14. The non-transitory computer-readable apparatus of claim 8 , wherein the processing of each of the plurality of portions of the generated content comprises application of the at least one respective encoding parameter to encode each of the plurality of portions of the generated content.

15. A method of processing digital image data, the method comprising:
obtaining visual content having a plurality of spatial portions;
processing each of the plurality of spatial portions of the visual content based on one or more encoding quality parameters, each of the one or more encoding quality parameters being specific to respective ones of each of the plurality of spatial portions; and
generating derivative content based at least on the processing of each of the plurality of spatial portions of the visual content, the derivative content having (i) one or more areas associated with a first quality level, and (ii) one or more areas associated with a second quality level that is greater than the first quality level.

16. The method of claim 15 , wherein the encoding quality parameters are described via at least a distribution function, the distribution function relating encoding quality to position within a coordinate system associated with the obtained visual content.

17. The method of claim 16 , further comprising:
adjusting the encoding quality parameters, the adjusting the encoding quality parameters resulting in at least an adjustment of the distribution function to produce an adjusted distribution function; and
generating second derivative content based at least on the adjusted distribution function.

18. The method of claim 15 , further comprising determining at least some of the encoding quality parameters based on a time of retrieval of the at least some of the encoding quality parameters.

19. The method of claim 15 , wherein the generating of the derivative content is further based on a rendering capability associa

Assignees

• Gopro Inc

Inventors

• Adsumilli Balineedu Chowdary
• Abbas Adeel
• Chawla Sumit

Classifications

• H04N19/597 Primary

  specially adapted for multi-view video sequence encoding · CPC title

• H04N23/698 Primary

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• H04N19/117

  Filters, e.g. for pre-processing or post-processing (sub-band filter banks H04N19/635) · CPC title

• H04N19/122

  Selection of transform size, e.g. 8x8 or 2x4x8 DCT; Selection of sub-band transforms of varying structure or type · CPC title

• H04N13/178

  Metadata, e.g. disparity information · CPC title