Artistic intent based video coding

Patent metadata

Field	Value
Publication number	US-11323722-B2
Application number	US-201615567966-A
Country	US
Kind code	B2
Filing date	Apr 21, 2016
Priority date	Apr 21, 2015
Publication date	May 3, 2022
Grant date	May 3, 2022

Abstract

Expressions of artistic intent are identified (e.g. by signaling or content analysis) and expressed as a set of artistic intent positions P.sub.ART and artistic intent characteristics C.sub.ART, Artistic intent characteristics C.sub.ART may be signaled and used to identify artistic intent positions P.sub.ART. Artistic intent preservation coding and processing may be applied to sample positions P.sub.ART to preserve characteristics C.sub.ART. A coding user interface may permit a user to specify an artistic set (e.g. P.sub.ART and/or C.sub.ART) and to select and/or configure treatment of pixels and/or blocks associated with an artistic set, such as a fidelity enhancement, QP adjustment value and/or postprocessing. Content priority or importance levels may be impliedly and/or expressly indicated at fine (e.g. pixel, sample) and/or coarse (e.g. block) levels of content for varying (e.g. enhanced, reduced) levels of treatment in content coding, delivery, processing and/or error resilience/robustness.

First claim

What is claimed is:

1. A method comprising:
decoding from a bitstream sample values associated with sample locations of an image;
receiving in the bitstream an indication of one or more characteristic sets, each characteristic set comprising a characteristic type, the characteristic type being one in a plurality of characteristic types;
receiving in the bitstream an indication of a description of the one or more characteristic sets, the description comprising characteristic values associated with the one or more characteristic sets;
determining a subset of the sample locations, the subset of sample locations having characteristic values corresponding to characteristic values associated with the one or more characteristic sets; and
determining, based at least on the subset of sample locations having characteristic values corresponding to characteristic values associated with the one or more characteristic sets, to apply modified processing to the subset of the sample locations.

2. The method of claim 1 , wherein the characteristic type comprises one of luminance, chrominance, or chromaticity.

3. The method of claim 1 , wherein the characteristic type comprises one of depth or contrast.

4. The method of claim 1 , wherein the characteristic type comprises one of an edge property or a texture property.

5. The method of claim 4 , wherein the edge property is one of edge presence, edge strength, or edge orientation.

6. The method of claim 1 , wherein the description of the one or more characteristic sets identifies a range of values in a 1-D characteristic value space.

7. The method of claim 1 , wherein the description of the one or more characteristic sets identifies a shape enclosing values in a 2-D characteristic value space or a volume of values in a 3-D characteristic value space.

8. The method of claim 1 , wherein determining to apply modified processing to the subset of sample locations comprises determining to apply one or more of a quantization step size adjustment, a modified loop filter, a modified post-processing filter, a modified deblocking filter, or a cross- component filter.

9. The method of claim 1 ,
wherein the one or more characteristic sets comprises at least two characteristic sets, and
wherein determining the subset of the sample locations comprises determining the subset of the sample locations that have characteristic values corresponding to characteristic values associated with the at least two characteristic sets.

10. The method of claim 1 , further comprising determining from the bitstream an importance level associated with the one or more characteristic sets,
wherein determining to apply modified processing to the subset of sample locations comprises determining to apply modified processing based on the importance level and the subset of sample locations having characteristic values corresponding to characteristic values associated with the one or more characteristic sets.

11. A computing system, comprising:
a processor programed to perform operations comprising:
decoding from a bitstream sample values associated with sample locations of an image;
receiving in the bitstream an indication of one or more characteristic sets, each characteristic set comprising a characteristic type, the characteristic type being one in a plurality of characteristic types;
receiving in the bitstream an indication of a description of the one or more characteristic sets, the description comprising characteristic values associated with the one or more characteristic sets;
determining a subset of the sample locations, the subset of sample locations having characteristic values corresponding to characteristic values associated with the one or more characteristic sets; and
determining, based at least on the subset of sample locations having characteristic values corresponding to characteristic values associated with the one or more characteristic sets, to apply modified processing to the subset of the sample locations.

12. The computing system of claim 11 , wherein the characteristic type comprises one of luminance, chrominance, or chromaticity.

13. The computing system of claim 11 , wherein the characteristic type comprises one of depth or contrast.

14. The computing system of claim 11 , wherein the characteristic type comprises one of an edge property or a texture property.

15. The computing system of claim 14 , wherein the edge property is one of edge presence, edge strength, or edge orientation.

16. The computing system of claim 11 , wherein the description of the one or more characteristic sets identifies a range of values in a 1-D characteristic value space.

17. The computing system of claim 11 , wherein the description of the one or more characteristic sets identifies a shape enclosing values in a 2-D characteristic value space or a volume of values in a 3-D characteristic value space.

18. The computing system of claim 11 , wherein determining to apply modified processing to the subset of sample locations comprises determining to apply one or more of a quantization step size adjustment, a modified loop filter, a modified post-processing filter, a modified deblocking filter, or a cross-component filter.

19. The computing system of claim 11 , wherein the one or more characteristic sets comprises at least two characteristic sets, and wherein determining the subset of the sample locations comprises determining the subset of the sample locations that have characteristic values corresponding to characteristic values associated with the at least two characteristic sets.

20. The computing system of claim 11 , wherein the processor is programmed to perform further operations comprising determining from the bitstream an importance level associated with the one or more characteristic sets,
wherein determining to apply modified processing to the subset of sample locations comprises determining to apply modified processing based on the importance level and the subset of sample locations having characteristic values corresponding to characteristic values associated with the one or more characteristic sets.

21. A non-transitory computer readable medium having stored thereon instructions executable by one or more processors for performing operations comprising:
decoding from a bitstream sample values associated with sample locations of an image;
receiving in the bitstream an indication of one or more characteristic sets, each characteristic set comprising a characteristic type, the characteristic type being one in a plurality of characteristic types;
receiving in the bitstream an indication of a description of the one or more characteristic sets, the description comprising characteristic values associated with the one or more characteristic sets;
determining a subset of the sample locations, the subset of sample locations having characteristic values corresponding to characteristic values associated with the one or more characteristic sets; and
determining, based at least on the subset of sample locations having characteristic values corresponding to characteristic values associated with the one or more characteristic sets, to apply modified processing to the subset of the sample locations.

22. The non-transitory computer readable medium of claim 21 , wherein the characteristic type comprises one of luminance, chrominance, or chromaticity.

23. The non-transitory computer readable medium of claim 21 , wherein the characteristic type comprises one of d

Assignees

• Interdigital Madison Patent Holdings Sas

Inventors

• Yi Byung K
• Ye Yan
• He Yuwen
• Xiu Xiaoyu
• He Yong
• Neff Ralph

Classifications

• H04N19/115 Primary

  Selection of the code volume for a coding unit prior to coding · CPC title

• H04N19/167 Primary

  Position within a video image, e.g. region of interest [ROI] · CPC title

• H04N19/124

  Quantisation · CPC title

• H04N19/14

  Coding unit complexity, e.g. amount of activity or edge presence estimation (H04N19/146 takes precedence) · CPC title

• H04N19/17

  the unit being an image region, e.g. an object · CPC title