User input back channel for wireless displays

Patent metadata

Field	Value
Publication number	US-9582239-B2
Application number	US-201213344424-A
Country	US
Kind code	B2
Filing date	Jan 5, 2012
Priority date	Jan 21, 2011
Publication date	Feb 28, 2017
Grant date	Feb 28, 2017

Abstract

As part of a communication session, a wireless source device can transmit audio and video data to a wireless sink device, and the wireless sink device can transmit user input data received at the wireless sink device back to the wireless source device. In this manner, a user of the wireless sink device can control the wireless source device and control the content that is being transmitted from the wireless source device to the wireless sink device. The input data received at the wireless sink device can have associated coordinate information that is scaled or normalized by either the wireless sink device or the wireless source device.

First claim

The invention claimed is:

1. A method of transmitting user data from a wireless sink device to a wireless source device, the method comprising:
receiving from the wireless source device, an indication of a resolution of the wireless source device;
obtaining user input data at the wireless sink device, wherein the user input data has associated coordinate data;
at the wireless sink device, normalizing the associated coordinate data based on the resolution of the wireless source device and a resolution of the wireless sink device to generate normalized coordinate data;
generating a data packet comprising a data packet header and payload data, wherein the data packet header includes a value for an input category field indicating that the payload data comprises generic information elements, wherein the generic information elements describe the user input data according to an application-level protocol being executed by the wireless source device and the wireless sink device, and wherein the payload data includes one or more generic information elements comprising the normalized coordinate data, wherein the value is selected from a list of supported input categories, the list of supported input categories comprising the value for the generic input and a value indicating the payload data is formatted according to a type of human interface device command used to receive the input data; and
transmitting the data packet to a wireless source device from the wireless sink device.

2. The method of claim 1 , further comprising:
determining if the associated coordinate data is within a display window for content being received from the wireless source device.

3. The method of claim 1 , further comprising:
determining a resolution of a display window for content being received from the wireless source device.

4. The method of claim 3 , wherein normalizing the coordinate data comprises scaling the associated coordinate data based on a ratio of the resolution of the display window and the resolution of the display of the source.

5. The method of claim 1 , wherein the associated coordinate data corresponds to a location of a mouse click event.

6. The method of claim 1 , wherein the associated coordinate data corresponds to a location of a touch event.

7. A wireless sink device for transmitting user data to a wireless source device, the wireless sink device comprising:
a memory storing instructions;
one or more processors configured to execute the instructions, wherein upon execution of the instructions the one or more processors cause:
receiving from the wireless source device, an indication of a resolution of the wireless source device;
obtaining user input data at the wireless sink device, wherein the user input data has associated coordinate data;
normalizing the associated coordinate data based on the resolution of the wireless source device and a resolution of the wireless sink device to generate normalized coordinate data; and
generating a data packet comprising a data packet header and payload data, wherein the data packet header includes a value for an input category field indicating that the payload data comprises generic information elements, wherein the generic information elements describe the user input data according to an application-level protocol being executed by the wireless source device and the wireless sink device, and wherein the payload data includes one or more generic information elements comprising the normalized coordinate data, wherein the value is selected from a list of supported input categories, the list of supported input categories comprising the value for the generic input and a value indicating the payload data is formatted according to a type of human interface device command used to receive the input data;

a transport unit to transmit the data packet to the wireless source device.

8. The wireless sink device of claim 7 , wherein upon execution of the instructions the one or more processors further cause:
determining if the associated coordinate data is within a display window for content being received from the wireless source device.

9. The wireless sink device of claim 7 , wherein upon execution of the instructions the one or more processors further cause:
determining a resolution of a display window for content being received from the wireless source device.

10. The wireless sink device of claim 9 , wherein normalizing the coordinate data comprises scaling the associated coordinate data based on a ratio of the resolution of the display window and the resolution of the display of the source.

11. The wireless sink device of claim 7 , wherein the associated coordinate data corresponds to a location of a mouse click event.

12. The wireless sink device of claim 7 , wherein the associated coordinate data corresponds to a location of a touch event.

13. A non-transitory, computer-readable storage medium storing instructions that upon execution by one or more processors cause the one or more processors to:
receive from a wireless source device, an indication of a resolution of the wireless source device;
obtain user input data at a wireless sink device, wherein the user input data has associated coordinate data;
at the wireless sink device, normalize the associated coordinate data based on the resolution of the wireless source device and a resolution of the wireless sink device to generate normalized coordinate data;
generate a data packet comprising a data packet header and payload data, wherein the data packet header includes a value for an input category field indicating that the payload data comprises generic information elements, wherein the generic information elements describe the user input data according to an application-level protocol being executed by the wireless source device and the wireless sink device, and wherein the payload data includes one or more generic information elements comprising the normalized coordinate data, wherein the value is selected from a list of supported input categories, the list of supported input categories comprising the value for the generic input and a value indicating the payload data is formatted according to a type of human interface device command used to receive the input data; and
transmit the data packet to a wireless source device.

14. The non-transitory computer-readable storage medium of claim 13 , storing further instructions that when executed by the one or more processors cause the one or more processors to:
determine if the associated coordinate data is within a display window for content being received from the wireless source device.

15. The non-transitory computer-readable storage medium of claim 13 , storing further instructions that when executed by the one or more processors cause the one or more processors to:
determine a resolution of a display window for content being received from the wireless source device.

16. The non-transitory computer-readable storage medium of claim 13 , wherein to normalize the coordinate data, the one or more processors scale the associated coordinate data based on a ratio of the resolution of the display window and the resolution of the display of the source.

17. The non-transitory computer-readable storage medium of claim 13 , wherein the associated coordinate data corresponds to a location of a mouse click event.

18. The non-transitory computer-readable storage medium of claim 13 , wherein the associated coordinate data corresponds to a location of a touch event.

19. A wireless sink device for transmit

Assignees

• Raveendran Vijayalakshmi R
• Huang Xiaolong
• Wang Xiaodong
• Shaukat Fawad
• Qualcomm Inc

Inventors

• Raveendran Vijayalakshmi R
• Huang Xiaolong
• Wang Xiaodong
• Shaukat Fawad

Classifications

• G09G2370/06

  Consumer Electronics Control, i.e. control of another device by a display or vice versa · CPC title

• G09G2370/16

  Use of wireless transmission of display information · CPC title

• G06F3/1454 Primary

  involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay · CPC title

• G09G2370/10

  Use of a protocol of communication by packets in interfaces along the display data pipeline · CPC title