User input back channel for wireless displays

Patent metadata

Field	Value
Publication number	US-10135900-B2
Application number	US-201213344273-A
Country	US
Kind code	B2
Filing date	Jan 5, 2012
Priority date	Jan 21, 2011
Publication date	Nov 20, 2018
Grant date	Nov 20, 2018

Abstract

As part of a communication session, a wireless source device can transmit audio and video data to a wireless sink device, and the wireless sink device can transmit user inputs received at the wireless sink device back to the wireless source device. In this manner, a user of the wireless sink device can control the wireless source device and control the content that is being transmitted from the wireless source device to the wireless sink device. As part of transmitting user input to the source device, the sink device can generate a data packet with a data packet header that includes a timestamp flag indicating the presence of a timestamp field, and the source device can process the data packet based on a timestamp in the timestamp field.

First claim

The invention claimed is:

1. A method of transmitting user input data from a wireless sink device to a wireless source device, the method comprising:
receiving, by the wireless sink device from the wireless source device, video data comprising a plurality of frames of video data, wherein each frame of the plurality of frames of video data comprises a respective timestamp;
in response to a user input event, obtaining user input data by the wireless sink device while displaying the plurality of frames of video data, wherein the user input data comprises coordinate data that identifies a location associated with the user input event, wherein the location is a location in a frame of video data being displayed by the wireless sink device when the user input data was obtained;
responsive to obtaining the user input data, generating a data packet header comprising a timestamp field that specifies one of the respective timestamps, the one of the respective timestamps being a timestamp of the frame video data being displayed by the wireless sink device when the user input data was obtained;
responsive to obtaining the user input data and generating the data packet header, generating a data packet comprising the data packet header and payload data, wherein the payload data comprises the coordinate data; and
transmitting the data packet from the wireless sink device to the wireless source device via a user input back channel (UIBC).

2. The method of claim 1 , wherein the timestamp is based on multimedia data that was generated by the wireless source device and transmitted to the wireless sink device.

3. The method of claim 1 , further comprising:
setting a timestamp flag of the data packet header to indicate that the timestamp field is present in the data packet header.

4. The method of claim 1 , further comprising:
generating a second data packet header, wherein a timestamp flag of the second data packet header indicates a timestamp field for the second data packet header is not present in the second data packet header;
generating a second data packet comprising the second data packet header, wherein the second data packet header does not contain a timestamp field.
transmitting the second data packet to the wireless source device.

5. The method of claim 1 , wherein the data packet header is an application layer packet header.

6. The method of claim 1 , wherein the data packet is to control audio data or video data of the wireless source device.

7. The method of claim 1 , wherein the data packet header further comprises a field to identify an input category of the user input data.

8. The method of claim 7 , wherein the input category is selected from the group consisting of a generic command and an operating system (OS) specific command.

9. The method of claim 1 , wherein the data packet header further comprises a field identifying a version of a protocol being used for communication.

10. The method of claim 1 , wherein the data packet header further comprises a length field to indicate a length of the data packet.

11. The method of claim 10 , wherein the length is identified in units of 16 bits.

12. The method of claim 1 , wherein the data packet is transmitted using transmission control protocol and internet protocol (TCP/IP).

13. The method of claim 3 , wherein the timestamp flag is a flag and the timestamp field is a 16-bit field.

14. A wireless sink device configured to transmit user input data to a wireless source device, the wireless sink device comprising:
a receiver unit configured to receive video data comprising a plurality of frames of video data, wherein each frame of the plurality of frames of video data comprises a respective timestamp;
a memory storing instructions;
one or more processors configured to execute the instructions, wherein upon execution of the instructions the one or more processors cause the wireless sink device to:
in response to a user input event, obtain user input data while displaying the frames of the video data received from the wireless source device, wherein the user input data comprises coordinate data that identifies a location associated with the user input event, wherein the location is a location in a frame of video data being displayed by the wireless sink device when the user input data was obtained;
responsive to obtaining the user input data, generate a data packet header comprising a timestamp field that specifies one of the respective timestamps, the one of the respective timestamps being a timestamp of the frame of video data being displayed by the wireless sink device when the user input data was obtained; and;
responsive to obtaining the user input data and generating the data packet header, generate a data packet comprising the data packet header and payload data, wherein the payload data comprises the coordinate data; and

a transmitter unit to transmit the data packet to the wireless source device via a user input back channel (UIBC).

15. The wireless sink device of claim 14 , wherein the timestamp is based on multimedia data that was generated by the wireless source device and transmitted to the wireless sink device.

16. The wireless sink device of claim 14 , wherein the one or more processors further cause the wireless sink device to:
set a timestamp flag of the data packet header to indicate that the timestamp field is present in the data packet header.

17. The wireless sink device of claim 14 , wherein the one or more processors further cause the wireless sink device to:
generate a second data packet header, wherein a timestamp flag of the second data packet header indicates a timestamp field for the second data packet header is not present in the second data packet header;
generate a second data packet comprising the second data packet header, wherein the second data packet header does not contain a timestamp field.
transmit the second data packet to the wireless source device.

18. The wireless sink device of claim 14 , wherein the data packet header is an application layer packet header.

19. The wireless sink device of claim 14 , wherein the data packet is to control audio data or video data of the wireless source device.

20. The wireless sink device of claim 14 , wherein the data packet header further comprises a field to identify an input category of the user input data.

21. The wireless sink device of claim 20 , wherein the input category is selected from the group consisting of a generic command and an operating system (OS) specific command.

22. The wireless sink device of claim 14 , wherein the data packet header further comprises a field identifying a version of a protocol being used for communication.

23. The wireless sink device of claim 14 , wherein the data packet header further comprises a length field to indicate a length of the data packet.

24. The wireless sink device of claim 23 , wherein the length is identified in units of 16 bits.

25. The wireless sink device of claim 14 , wherein the data packet is transmitted using transmission control protocol and internet protocol (TCP/IP).

26. The wireless sink device of claim 16 , wherein the timestamp flag is a 1-bit flag and the timestamp field is a 16-bit field.

27. A non-transitory computer-readable storage medium storing instructions that upon execution by one or more processors cause the one or more processors to:
receive a plurality of fram

Assignees

• Raveendran Vijayalakshmi R
• Huang Xiaolong
• Wang Xiaodong
• Shaukat Fawad
• Qualcomm Inc

Inventors

• Raveendran Vijayalakshmi R
• Huang Xiaolong
• Wang Xiaodong
• Shaukat Fawad

Classifications

• H04L65/608 Primary

  Electricity · mapped topic

• H04L65/756

  adapting media to device capabilities · CPC title

• H04W4/00

  Services specially adapted for wireless communication networks; Facilities therefor · CPC title

• H04W76/10 Primary

  Connection setup · CPC title

• H04L65/65 Primary

  Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP] · CPC title