Intelligent metadata service for video enhancement

Patent metadata

Field	Value
Publication number	US-11544826-B2
Application number	US-202117245814-A
Country	US
Kind code	B2
Filing date	Apr 30, 2021
Priority date	Apr 30, 2021
Publication date	Jan 3, 2023
Grant date	Jan 3, 2023

Abstract

Embodiments are directed towards metadata service for video enhancement. An example method includes receiving a request for metadata applicable to instruct a display device to transform low dynamic range (LDR) video content to high dynamic range (HDR) video content, determining availability of candidate metadata, and responsive to determining that no candidate metadata is available, providing options for generating the metadata applicable to instruct the display device. The example method can also include causing generation of the metadata based at least one of the options, and providing the generated metadata to the display device.

First claim

The invention claimed is:

1. A system, comprising:
memory that stores computer instructions; and
at least one processor that executes the computer instructions to perform actions, the actions comprising:
receiving, from a first user device, a request for metadata applicable to instruct the first user device to transform target low dynamic range (LDR) video content to high dynamic range (HDR) video content;
determining availability of candidate metadata indicating at least inverse tone map (ITM) parameters based, at least in part, on the request;
responsive to determining that no candidate metadata indicating at least the ITM parameters is available based, at least in part, on the request, providing options for generating the metadata applicable to instruct the first user device to transform the target LDR video content to HDR video content;
causing generation of the metadata indicating at least the ITM parameters based, at least in part, on at least one of the options; and
providing the generated metadata to the first user device.

2. The system of claim 1 , wherein the HDR video content is superior to the target LDR video content in at least one of resolution, framerate, or color dynamic range.

3. The system of claim 1 , wherein the at least one of the options includes generating the metadata by the system.

4. The system of claim 1 , wherein the at least one of the options includes causing a second user device to generate the metadata.

5. The system of claim 4 , wherein the actions further comprise receiving, from a third user device, a request for metadata applicable to instruct the third user device to transform the target LDR video content to HDR video content.

6. The system of claim 5 , wherein the actions further comprise providing the generated metadata to the third user device.

7. The system of claim 1 , wherein providing the generated metadata to the first user device comprises causing streaming of the generated metadata to the first user device separately from a stream of the target LDR video content.

8. A method, comprising:
receiving, from a first user device, a request for metadata applicable to instruct the first user device to enhance target video content;
determining availability of candidate metadata indicating at least inverse tone map (ITM) parameters based, at least in part, on the request;
responsive to determining that no candidate metadata indicating at least the ITM parameters is available based, at least in part, on the request, providing options for generating the metadata applicable to instruct the first user device to enhance the target video content;
causing generation of the metadata indicating at least the ITM parameters based, at least in part, on at least one of the options; and
providing the generated metadata to the first user device.

9. The method of claim 8 , wherein determining availability of candidate metadata is further based on at least one of communication capability, computational capability, video rendering capability, or display capability of the first user device.

10. The method of claim 8 , wherein determining availability of candidate metadata comprises identifying a set of metadata provided by one or more second user devices.

11. The method of claim 8 , wherein determining availability of candidate metadata comprises identifying a set of metadata generated in response to one or more previously received requests for metadata.

12. The method of claim 8 , wherein causing generation of the metadata comprises combining metadata applicable to different portions of the target video content.

13. The method of claim 8 , wherein causing generation of the metadata comprises selecting one or more portions of metadata applicable to video content that at least partially overlap with the target video content.

14. The method of claim 8 , wherein causing generation of the metadata comprises causing one or more second user devices each to generate at least a portion of the metadata.

15. The method of claim 14 , wherein the one or more second user devices are compatible with the first user device in at least one of communication capability, computational capability, or video rendering capability, or display capability.

16. A non-transitory computer-readable storage medium storing contents that, when executed by one or more processors, cause the one or more processors to:
receive, from a first user device, a request for metadata applicable to instruct the first user device to enhance target video content;
determine availability of candidate metadata indicating at least inverse tone map (ITM) parameters based, at least in part, on the request;
responsive to determining that no candidate metadata indicating at least the ITM parameters is available based, at least in part, on the request, provide options for generating the metadata applicable to instruct the first user device to enhance the target video content;
cause generation of the metadata indicating at least the ITM parameters based, at least in part, on at least one of the options; and
provide the generated metadata to the first user device.

17. The storage medium of claim 16 , wherein providing the generated metadata to the first user device comprises causing streaming of the generated metadata to the first user device separately from a stream of the target video content.

18. The storage medium of claim 16 , wherein the contents further cause the one or more processors to receive, from a second user device, a request for metadata applicable to instruct the second user device to enhance other video content that at least partially overlaps with the target video content.

19. The storage medium of claim 18 , wherein the contents further cause the one or more processors to provide at least a portion of the generated metadata to the second user device.

20. The storage medium of claim 16 , wherein determining availability of candidate metadata comprises identifying a set of metadata provided by one or more second user devices.

Assignees

• Realnetworks Inc

Inventors

• Rassool Reza

Classifications

• G06T2207/10016

  Video; Image sequence · CPC title

• H04N21/84

  Generation or processing of descriptive data, e.g. content descriptors {(systems specially adapted for using meta-information in broadcast systems H04H60/73)} · CPC title

• H04N21/435

  Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream · CPC title

• G06T2207/20208

  High dynamic range [HDR] image processing · CPC title

• H04N21/4402 Primary

  involving reformatting operations of video signals for household redistribution, storage or real-time display {(details of conversion of video standards at pixel level H04N7/01; video transcoding H04N19/40; adapting incoming signals to the display format of the display terminal G09G5/005; media handling at the source in data packet switching networks H04L65/764)} · CPC title