Context based adaptive resolution modulation countering network latency fluctuation

Patent metadata

Field	Value
Publication number	US-11653047-B2
Application number	US-202117388439-A
Country	US
Kind code	B2
Filing date	Jul 29, 2021
Priority date	Jul 29, 2021
Publication date	May 16, 2023
Grant date	May 16, 2023

Abstract

A method includes communicating a first stream of a video comprising first and second objects to a device. The first stream has a first resolution. The method also includes communicating a second stream to the device. The second stream indicates that the first object is contextual and that the second object is non-contextual. The method further includes, after a decrease in bandwidth, communicating a third stream of the video to the device. The third stream has a second resolution that is lower than the first resolution. When the video is presented for display using the third stream and based on the second stream indicating that the first object is contextual and that the second object is non-contextual, the first object is presented in the first resolution and the second object is presented in the second resolution.

First claim

What is claimed is:

1. A method comprising:
communicating a first stream of a video comprising first and second objects to a device, wherein the first stream has a first resolution;
communicating a second stream to the device, wherein the second stream indicates that the first object is contextual and that the second object is non-contextual; and
after a decrease in bandwidth, communicating a third stream of the video to the device, wherein the third stream has a second resolution that is lower than the first resolution, and wherein, when the video is presented for display using the third stream and based on the second stream indicating that the first object is contextual and that the second object is non-contextual, the first object is presented in the first resolution and the second object is presented in the second resolution.

2. The method of claim 1 , wherein the second stream comprises the first object in the first resolution.

3. The method of claim 1 , further comprising generating a dependency map that indicates that a third object is non-contextual in a first scene of the video and that the third object is contextual in a second scene of the video, wherein the second scene occurs after the first scene, and wherein the first scene and the second scene are presented when the video is presented for display using the third stream.

4. The method of claim 3 , wherein, based on the dependency map indicating that the third object is contextual in the second scene, the third object is presented in the first resolution when the first scene is presented.

5. The method of claim 3 , wherein:
the dependency map further indicates that the first object is non-contextual in the second scene; and
based on the dependency map indicating that the first object is non-contextual in the second scene, the first object is presented in the second resolution when the second scene is presented.

6. The method of claim 3 , wherein the second stream is generated based on the dependency map and wherein the second stream indicates that the third object is non-contextual in the first scene and contextual in the second scene.

7. The method of claim 1 , further comprising applying a machine learning process to the video to determine that the first object is contextual and that the second object is non-contextual.

8. An apparatus comprising:
a memory; and
a hardware processor communicatively coupled to the memory, the hardware processor configured to:
communicate a first stream of a video comprising first and second objects to a device, wherein the first stream has a first resolution;
communicate a second stream to the device, wherein the second stream indicates that the first object is contextual and that the second object is non-contextual; and
after detecting a decrease in bandwidth, communicate a third stream of the video to the device, wherein the third stream has a second resolution that is lower than the first resolution, and wherein, when the video is presented for display using the third stream and based on the second stream indicating that the first object is contextual and that the second object is non-contextual, the first object is presented in the first resolution and the second object is presented in the second resolution.

9. The apparatus of claim 8 , wherein the second stream comprises the first object in the first resolution.

10. The apparatus of claim 8 , the hardware processor further configured to generate a dependency map that indicates that a third object is non-contextual in a first scene of the video and that the third object is contextual in a second scene of the video, wherein the second scene occurs after the first scene, and wherein the first scene and the second scene are presented when the video is presented for display using the third stream.

11. The apparatus of claim 10 , wherein, based on the dependency map indicating that the third object is contextual in the second scene, the third object is presented in the first resolution when the first scene is presented.

12. The apparatus of claim 10 , wherein:
the dependency map further indicates that the first object is non-contextual in the second scene; and
based on the dependency map indicating that the first object is non-contextual in the second scene, the first object is presented in the second resolution when the second scene is presented.

13. The apparatus of claim 10 , wherein the second stream is generated based on the dependency map and wherein the second stream indicates that the third object is non-contextual in the first scene and contextual in the second scene.

14. The apparatus of claim 8 , the hardware processor further configured to apply a machine learning process to the video to determine that the first object is contextual and that the second object is non-contextual.

15. A method comprising:
receiving, at a device, a first stream of a video comprising first and second objects, wherein the first stream has a first resolution;
receiving, at the device, a second stream that indicates that the first object is contextual and that the second object is non-contextual;
after a decrease in bandwidth, receiving, at the device, a third stream of the video, wherein the third stream has a second resolution that is lower than the first resolution; and
based on the second stream indicating that the first object is contextual and that the second object is non-contextual and when displaying the video using the third stream, displaying the first object in the first resolution and the second object in the second resolution.

16. The method of claim 15 , wherein the second stream comprises the first object in the first resolution.

17. The method of claim 15 , wherein a first scene of the video and a second scene of the video are displayed when displaying the video using the third stream, wherein the second scene occurs after the first scene, and wherein, based on a dependency map indicating that a third object is non-contextual in the first scene and that the third object is contextual in the second scene, displaying the third object in the first resolution when displaying the first scene.

18. The method of claim 17 , wherein:
the dependency map further indicates that the first object is non-contextual in the second scene; and
based on the dependency map indicating that the first object is non-contextual in the second scene, displaying the first object in the second resolution when displaying the second scene.

19. The method of claim 17 , wherein the second stream is generated based on the dependency map and wherein the second stream indicates that the third object is non-contextual in the first scene and contextual in the second scene.

20. The method of claim 15 , wherein a machine learning process is applied to the video to determine that the first object is contextual and that the second object is non-contextual.

Assignees

• IBM

Inventors

• Pandey Diwesh
• Bose Tathagato
• Joseph Arun

Classifications

• H04N19/20 Primary

  using video object coding · CPC title

• H04N21/2405

  Monitoring of the internal components or processes of the server, e.g. server load (allocation of resources in multiprogramming arrangements G06F9/50; performance measurement of computer activity G06F11/34) · CPC title

• H04N21/2402

  Monitoring of the downstream path of the transmission network, e.g. bandwidth available (traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876) · CPC title

• H04N19/17

  the unit being an image region, e.g. an object · CPC title

• H04N21/2662 Primary

  Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities · CPC title