Method for wirelessly transmitting content from a source device to a sink device

Patent metadata

Field	Value
Publication number	US-9412332-B2
Application number	US-201314136944-A
Country	US
Kind code	B2
Filing date	Dec 20, 2013
Priority date	Dec 20, 2013
Publication date	Aug 9, 2016
Grant date	Aug 9, 2016

Abstract

Methods and devices for wirelessly transmitting content from a source device to a sink device are disclosed. The method comprises: identifying one or more data types associated with a display frame displayed on the source device; selecting a transmission format for the display frame in accordance with the one or more identified data types, wherein the transmission format is selected from the group consisting of screencasting, graphics processing unit (GPU) processing, or GPU processing with media streaming; and sending visual information representing the display frame in the transmission selected transmission format to the sink device. One or a combination of latency, image/video quality, and power consumption associated with the wireless transmission may be used to adapt the wireless transmission.

First claim

The invention claimed is:

1. A method for wirelessly transmitting content from a source device to a sink device, comprising:
identifying, by a processor of the source device, one or more data types associated with a first display frame displayed on the source device, wherein the first display frame includes one or more data types selected from programmatically generated graphics and encoded media;
selecting, by the processor of the source device, a transmission format for the first display frame in accordance with the one or more identified data types,
wherein screencasting is selected when the first display frame includes only encoded media,
wherein graphics processing unit (GPU) processing is selected when the first display frame includes only programmatically generated graphics, and
wherein GPU processing with media streaming is selected when the first display frame includes both programmatically generated graphics and encoded media; and

preparing, by the processor and/or GPU of the source device, a second display frame for the sink device from the first display frame in accordance with the selected transmission format;
wirelessly transmitting, by a wireless communication system of the source device, the second display frame from the source device to the sink device,
determining, by the processor of the source device, whether predetermined performance criteria have been met;
when the predetermined performance criteria have not been met and the selected transmission format is GPU processing or GPU processing with media streaming, changing the transmission format from GPU processing or GPU processing with media streaming to screencasting.

2. The method of claim 1 , wherein the encoded media comprises images, video, or a combination of images and video.

3. The method of claim 1 , wherein the GPU processing complies with OpenGL or DirectX.

4. The method of claim 1 , wherein the screencasting complies with Miracast.

5. The method of claim 1 , wherein the media streaming complies with DLNA media streaming.

6. The method of claim 1 , further comprising:
when the selected transmission format is screencasting, the preparing comprises encoding the second display frame in a screencast encoding format.

7. The method of claim 1 , wherein, when the selected transmission format is GPU processing, programmatically generated graphics are sent as a set of one or more GPU display commands which reference one or more graphics resources.

8. The method of claim 7 , further comprising:
sending the one or more graphics resources referenced by the GPU display commands.

9. The method of claim 8 , wherein the one or more graphics resources are sent before sending any GPU display commands.

10. The method of claim 1 , wherein, when the selected transmission format is GPU processing with media streaming, portions of the first display frame comprising programmatically generated graphics are provided in the second display fame as a set of one or more GPU display commands which reference one or more graphics resources and portions of the first display fame which comprise encoded media are provided in the second display frame as encoded media objects.

11. The method of claim 1 , further comprising:
when the first display frame is associated with multiple data types, determining a portion of the first display frame associated with each data type and selecting a transmission format for transmitting each portion of the first display frame, wherein each portion of the first display frame is sent in the selected transmission format;
when the first display frame is associated with one data type, the selected transmission format is based on the data type associated with the first display frame.

12. The method of claim 1 , further comprising:
when the predetermined performance criteria have not been met and the selected transmission format is screencasting, reducing a frame rate of the screencasting.

13. The method of claim 12 , wherein the frame rate of the screencasting is reduced until the predetermined performance criteria are met.

14. The method of claim 1 , wherein the predetermined performance criteria comprise a transmission latency associated with sending the second display frame to the sink device, a processing latency associated with processing the second display frame on the sink device, a loss of quality, or any combination thereof.

15. The method of claim 1 , wherein the transmission format is selected in accordance with the one or more determined data types and a power consumption rating of each transmission format so that the lower power consumption rating is selected.

16. A wireless communication device comprising:
a processor;
a graphics processing unit (GPU) coupled to the processor;
a display coupled to the processor;
a wireless communication system coupled to the processor;
a memory coupled to the processor and the GPU and storing instructions which configure the wireless communication device to:
identify one or more data types associated with a first display frame displayed on the display, wherein the first display frame includes one or more data types selected from programmatically generated graphics and encoded media;
select a transmission format for the first display frame in accordance with the one or more identified data types,
wherein screencasting is selected when the first display frame includes only encoded media,
wherein GPU processing is selected when the first display frame includes only programmatically generated graphics, and
wherein GPU processing with media streaming is selected when the first display frame includes both programmatically generated graphics and encoded media; and

prepare a second display frame for a sink device from the first display frame in accordance with the selected transmission format;

wherein the wireless communication system wirelessly transmits the second display frame to the sink device,
wherein the processor determines whether predetermined performance criteria have been met;
when the predetermined performance criteria have not been met and the selected transmission format is GPU processing or GPU processing with media streaming, changing the transmission format from GPU processing or GPU processing with media streaming to screencasting.

17. A non-transitory machine readable medium having tangibly stored thereon executable instructions for execution by a wireless communication device, the wireless communication device comprising a processor, a graphics processing unit (GPU) coupled to the processor, a display coupled to the processor, and a wireless communication system coupled to the processor, wherein the executable instructions, when executed by the processor and/or GPU, cause the wireless communication device to:
identify one or more data types associated with a first display frame displayed on the display, wherein the first display frame includes one or more data types selected from programmatically generated graphics and encoded media;
select a transmission format for the first display frame in accordance with the one or more identified data types,
wherein screencasting is selected when the first display frame includes only encoded media,
wherein GPU processing is selected when the first display frame includes only programmatically generated graphics, and
wherein GPU processing with media streaming is selected when the first display frame includes both programmatically generated graphics and encoded media; and

prepare a second display frame for a sink device from the first display frame in accordance

Assignees

• Blackberry Ltd

Inventors

• Smadi Mohammed Nawaf
• Kezys Vytautas Robertas
• Theimer Wolfgang Michael

Classifications

• G06F3/1454 Primary

  involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay · CPC title

• H04L65/607

  Electricity · mapped topic

• H04L65/80

  Responding to QoS · CPC title

• H04L67/38

  Electricity · mapped topic

• H04L65/60

  Network streaming of media packets · CPC title