Detection and repainting of semi-transparent overlays

Patent metadata

Field	Value
Publication number	US-11245754-B2
Application number	US-202016985505-A
Country	US
Kind code	B2
Filing date	Aug 5, 2020
Priority date	May 4, 2018
Publication date	Feb 8, 2022
Grant date	Feb 8, 2022

Abstract

A server includes at least one processor configured to execute a media application to provide media streaming that includes at least one video stream and at least one overlay on the at least one video stream. A portion of the media streaming is redirected by providing a placeholder to indicate positioning geometry of the at least one video stream within a media window, with the placeholder to include the at least one overlay. The processor detects the placeholder and determines positioning geometry associated therewith. A color and an alpha blending factor of the at least one overlay is determined based on calculations involving different colors of the at least one underlay at different times.

First claim

That which is claimed:

1. A server comprising:
at least one processor configured to perform the following:
execute a media application to provide media streaming that includes at least one video stream and at least one overlay on the at least one video stream;
redirect a portion of the media streaming by providing a placeholder to indicate positioning geometry of the at least one video stream within a media window, with the placeholder to include the at least one overlay;
detect the placeholder and determine positioning geometry associated therewith; and
determine a color and an alpha blending factor of the at least one overlay based on calculations involving different colors of the at least one underlay at different times.

2. The server according to claim 1 wherein said processor is configured to provide the placeholder based on the following:
provide a first color for an underlay of the placeholder at a first time; and
provide a second color for the underlay of the placeholder at a second time.

3. The server according to claim 2 wherein said processor is configured to determine the color and the alpha blending factor of the at least one overlay based on calculations involving the first color for the underlay of the placeholder at the first time, and the second color for the underlay of the placeholder at the second time.

4. The server according to claim 1 wherein said processor is further configured to provide the at least one video stream, the positioning geometry of the at least one placeholder, and the color and the alpha blending factor of the at least one overlay to a client device for the client device to overlay the at least one video stream over the at least one placeholder within a displayed media window based on the positioning geometry, and to overlay the at least one overlay in the color and alpha blending factor associated therewith.

5. The server according to claim 1 wherein the color and the alpha blending factor for the at least one overlay is determined for each pixel in the placeholder.

6. The server according to claim 1 wherein the color and the alpha blending factor for the at least one overlay is determined based on the following equations:
(pixel color at time 1)=(1−alpha)*(underlay color at time 1)+alpha*(overlay color)

(pixel color at time 2)=(1−alpha)*(underlay color at time 2)+alpha*(overlay color)

where alpha=1−(pixel color at time 2−pixel color at time 1)/(underlay color at time 2−underlay color at time 1);
where overlay color=(pixel color−(1−alpha)*underlay color)/alpha;
where pixel color corresponds to a blended color of the underlay color and the at least one overlay color at time 1 or at time 2;
wherein alpha corresponds to the alpha blending factor; and
wherein the overlay color corresponds to the color of the at least one overlay.

7. The server according to claim 6 wherein said processor is further configured to vary the first and second colors to improve color contrast between the underlay color and the at least one overlay color.

8. The server according to claim 1 wherein the media application comprises a real-time media application providing real-time communications (RTC), and wherein said processor is further configured to redirect the portion of the real-time media communications based on injected JavaScript code for WebRTC API redirection.

9. The server according to claim 1 wherein said processor is configured to redirect the portion of the media streaming based on at least one of the following: DirectShow filter, DirectX Media Object (DMO) filter, Media Foundation filter, HTML5 video redirection module, Flash video redirection module, Browser Helper Object (BHO), browser extension.

10. The computing system according to claim 1 wherein said processor is further configured to perform the following:
generate non-accelerated graphics for the redirected portion of the media streaming; and
analyze the non-accelerated graphics for the at least one placeholder and the at least one overlay.

11. The method according to claim 1 wherein the media application comprises a real-time media application providing real-time communications (RTC), and wherein redirecting the portion of the real-time media communications is based on injected JavaScript code for WebRTC API redirection.

12. A method comprising:
executing a media application to provide media streaming that includes at least one video stream and at least one overlay on the at least one video stream;
redirecting a portion of the media streaming by providing a placeholder to indicate positioning geometry of the at least one video stream within a media window, with the placeholder to include the at least one overlay;
detecting the placeholder and determine positioning geometry associated therewith; and
determining a color and an alpha blending factor of the at least one overlay based on calculations involving different colors of the at least one underlay at different times.

13. The method according to claim 12 wherein providing the placeholder comprises:
providing a first color for an underlay of the placeholder at a first time; and
providing a second color for the underlay of the placeholder at a second time.

14. The method according to claim 13 wherein the calculations involving different colors of the at least one underlay at different times involve the first color for the underlay of the placeholder at the first time, and the second color for the underlay of the placeholder at the second time.

15. The method according to claim 12 further comprising providing the at least one video stream, the positioning geometry of the at least one placeholder, and the color and the alpha blending factor of the at least one overlay to a client device for the client device to overlay the at least one video stream over the at least one placeholder within a displayed media window based on the positioning geometry, and to overlay the at least one overlay in the color and alpha blending factor associated therewith.

16. The method according to claim 12 wherein the color and the alpha blending factor for the at least one overlay is determined for each pixel in the placeholder.

17. The method according to claim 12 wherein the color and the alpha blending factor for the at least one overlay is determined based on the following equations:
(pixel color at time 1)=(1−alpha)*(underlay color at time 1)+alpha*(overlay color)

(pixel color at time 2)=(1−alpha)*(underlay color at time 2)+alpha*(overlay color)

where alpha=1−(pixel color at time 2−pixel color at time 1)/(underlay color at time 2−underlay color at time 1);
where overlay color=(pixel color−(1−alpha)*underlay color)/alpha;
where pixel color corresponds to a blended color of the underlay color and the at least one overlay color at time 1 or at time 2;
wherein alpha corresponds to the alpha blending factor; and
wherein the overlay color corresponds to the color of the at least one overlay.

18. The method according to claim 17 further comprising varying the first and second colors to improve color contrast between the underlay color and the at least one overlay color.

19. A computing system comprising:
at least one video source to provide at least one video stream; and
a server comprising at least one processor configured to perform the following:
execute a media application to provide media streaming that includes at least one video stream and at least one overlay on the at least one video stream,
redirect a portion of the media streaming by providing a pla

Assignees

• Citrix Systems Inc

Inventors

• Vysotsky Vladimir
• Giebler Todd
• Momchilov Georgy
• Page James

Classifications

• H04L65/403

  Arrangements for multi-party communication, e.g. for conferences (data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15) · CPC title

• H04L67/104 Primary

  Peer-to-peer [P2P] networks · CPC title

• H04L61/2589 Primary

  over a relay server, e.g. traversal using relay for network address translation [TURN] · CPC title

• G06F9/452

  Remote windowing, e.g. X-Window System, desktop virtualisation (protocols for virtual reality H04L67/131) · CPC title

• H04L67/59

  Providing operational support to end devices by off-loading in the network or by emulation, e.g. when they are unavailable · CPC title