Low-complexity remote presentation session encoder using subsampling in color conversion space

Patent metadata

Field	Value
Publication number	US-9712847-B2
Application number	US-201113237859-A
Country	US
Kind code	B2
Filing date	Sep 20, 2011
Priority date	Sep 20, 2011
Publication date	Jul 18, 2017
Grant date	Jul 18, 2017

Abstract

An invention is disclosed for encoding and decoding data in a 4:4:4 subsampling scheme, using an encoder/decoder that is not configured to encode or decode data in 4:4:4. In embodiments, an encoder planararizes an input frame into three component frames in a 4:0:0 scheme. The encoder then encodes each component frame in the 4:0:0 scheme, and aggregates the encoded component frames into a bit stream. A decoder receives such a bit stream, and decodes it with a component not configured to decode data in 4:4:4. The decoder decodes the bit stream to produce a representation of the three component frames in 4:0:0, then aggregates the three component frames into a representation of the original frame in 4:4:4.

First claim

What is claimed:

1. A method, comprising:
converting a frame of remote presentation session data comprising graphical data from RGB format to YUV format to produce a YUV frame;
dividing the YUV frame into three separate frames comprising a Y component frame, a U component frame, and a V component frame;
converting the frame to the Y component frame, the U component frame, and the V component frame based on determining that the frame comprises graphical data indicative of text;
encoding each of the separate Y, U, and V component frames in a second subsampling scheme with an encoder that is configured to encode a frame in the second subsampling scheme but is not configured to encode a frame in a first sampling scheme, wherein the second subsampling scheme comprises subsampling a subset of values of a pixel of a frame, the first subsampling scheme comprises individually sampling each value of a pixel of a frame;
encoding the encoded Y, U, and V component frames into a remote presentation session protocol bit stream; and
sending the bit stream over a network to a client computer via a remote presentation session, the client computer decoding the bit stream to produce a representation of each of the Y, U, and V component frames, and combining the representation of each of the Y, U, and V component frames into a representation of the YUV frame that the client computer displays on a display device.

2. The method of claim 1 , wherein encoding each of the Y, U, and V component frames in the second subsampling scheme with the encoder comprises:
encoding each of the Y, U, and V component frames in a 4:0:0 subsampling scheme with a H.264 intra-encoder.

3. The method of claim 1 , wherein encoding each of the Y, U, and V component frames in the second subsampling scheme with the encoder comprises:
determining a level of intra prediction with which to encode each of the Y, U, and V component frames; and
intra prediction encoding each of the Y, U, and V component frames with the encoder comprises based on the level of intra prediction.

4. The method of claim 3 , wherein determining the level of intra prediction comprises:
determining the level of intra prediction based on an amount of available computational resources.

5. The method of claim 1 , further comprising:
determining an amount of quantizing to quantize the encoded Y, U, and V component frames; and
quantizing the encoded Y, U, and V component frames before encoding the encoded Y, U, and V component frames into the remote presentation session protocol bit stream.

6. The method of claim 5 , wherein determining the amount of quantizing to quantize the encoded Y, U, and V component frames comprises:
determining a first amount of quantizing to quantize the Y component frames, and a second amount of quantizing to quantize the U and V component frames, the first amount of quantizing being less quantizing than the second amount of quantizing.

7. The method of claim 5 , wherein determining the amount of quantizing to quantize the encoded Y, U, and V component frames comprises:
determining the amount of quantizing to quantize the encoded Y, U, and V component frames based on a target bit rate of the bit stream.

8. The method of claim 1 , further comprising:
receiving a second frame in RGB format;
determining not to convert the frame into a second Y component frame, a second U component frame, and a second V component frame based on the second frame being in a subsampling scheme that involves less number of samples than the first subsampling scheme;
converting the second frame to YUV format;
encoding the converted second frame with the encoder;
encoding the encoded second frame into the remote presentation session protocol bit stream; and
sending the bit stream to the computer via the remote presentation session, the computer displaying a representation of the second frame on the display device based on the bit stream.

9. The method of claim 1 , wherein the first subsampling scheme comprises a 4:4:4 subsampling scheme, and wherein the second subsampling scheme comprises a 4:2:2, a 4:2:0, or a 4:0:0 subsampling scheme.

10. The method of claim 1 , further comprising:
marking each of the encoded Y, U, and V component frames with a reference to the frame before sending the bit stream to the computer; and
sending the bit stream to the computer via the remote presentation session, the computer combining each of the Y, U, and V component frames into the YUV frame based on each of the Y, U, and V component frames having the same reference.

11. A system, comprising:
a processor; and
a memory communicatively coupled to the processor when the system is operational, the memory bearing processor-executable instructions that, when executed on the processor, cause the system at least to:
receive over a network a bit stream via a remote presentation session, the bit stream comprising separately encoded first, second, and third component frames of a divided frame of graphical data of remote presentation session data from a server computer, wherein dividing the frame to the first, second, and third component frames based on determining that the frame comprises graphical data indicative of text, each of the encoded first, second, and third component frames being in a second subsampling scheme, wherein the second subsampling scheme comprises subsampling a subset of values of a pixel of a frame;
decode each of the encoded first, second, and third component frames with a decoder that is configured to decode frames in the second subsampling scheme but not configured to decode frames in a first subsampling scheme, wherein the first subsampling scheme comprises individually sampling each value of a pixel of a frame;
combine the decoded first, second, and third component frames to produce a representation of the frame of remote presentation session data; and
display the representation of the frame of remote presentation session data on a display device of a client computer.

12. The system of claim 11 , wherein the memory further bears processor-executable instructions that, when executed on the processor, cause the system at least to:
convert the frame from a first color space to a second color space before displaying the frame on the display device.

13. The system of claim 12 , wherein the processor-executable instructions that, when executed on the processor, cause the system at least to convert the frame from the first color space to the second color space before displaying the frame on the display device further cause the system at least to:
convert the frame from YUV color space to RGB color space before displaying the frame on the display device.

14. The system of claim 11 , wherein the processor-executable instructions, that when executed on the processor cause the system at least to combine the first, second and third component frames to produce the frame in the first subsampling scheme further cause the system at least to:
combine the first, second, and third component frames through a logical OR operation.

15. The system of claim 11 , wherein the decoder not configured to decode frames in the first subsampling scheme is configured to decode frames in a 4:2:0 subsampling scheme, the first subsampling scheme comprising a 4:4:4 subsampling scheme.

16. The system of claim 11 , wherein the bit stream received via the remote presentation session is encapsulated with a remote presentation protocol, and wherein the processor-executable instructions that, when executed on the processor, cause the system at least to: receive the bit stream via t

Assignees

• Malladi Krishna Mohan
• Kumar B Anil
• Abdo Nadim Y
• Sankuratri Sridhar
• Microsoft Technology Licensing Llc

Inventors

• Malladi Krishna Mohan
• Kumar B Anil
• Abdo Nadim Y
• Sankuratri Sridhar

Classifications

• H04N19/136

  Incoming video signal characteristics or properties · CPC title

• H04N19/124

  Quantisation · CPC title

• H04N19/103

  Selection of coding mode or of prediction mode · CPC title

• H04N19/186

  the unit being a colour or a chrominance component · CPC title

• H04N19/88 Primary

  involving rearrangement of data among different coding units, e.g. shuffling, interleaving, scrambling or permutation of pixel data or permutation of transform coefficient data among different blocks · CPC title