Controlling resolution of encoded video

Patent metadata

Field	Value
Publication number	US-9307195-B2
Application number	US-201414184228-A
Country	US
Kind code	B2
Filing date	Feb 19, 2014
Priority date	Oct 22, 2013
Publication date	Apr 5, 2016
Grant date	Apr 5, 2016

Abstract

A user terminal for participating in video calls comprises: an encoder having a frame size, being the size in pixels at which it encodes frames of video; and a pre-processing stage which supplies a sequence of frames to the encoder at that frame size, each frame comprising at least an image region representing a source video image at a respective moment in time. The pre-processing stage is configured to supply at least some of the frames to the encoder in a modified form, by resizing the source video image to produce the image region of each modified frame with a size smaller than the frame size of the encoder, and combining with a border region such that the modified frame matches the frame size of the encoder. The encoder encodes the frames at the frame size and transmits them to a receiving terminal as part of a live video call.

First claim

The invention claimed is:

1. A user terminal for participating in video calls, comprising:
an encoder having a frame size, the frame size being a size in pixels at which the encoder encodes frames of video; and
a pre-processing stage configured to supply a sequence of frames to the encoder at said frame size, each frame comprising at least an image region representing a source video image at a respective moment in time;
wherein the pre-processing stage is configured to supply at least some of said frames to the encoder in a modified form, by resizing the source video image to produce the image region of each modified frame with a size smaller than the frame size of the encoder, and combining with a border region such that the modified frame matches the frame size of the encoder, the resizing comprising cropping a portion of the source video image and replacing the cropped portion of the source video image with a portion of the border region; and
wherein the encoder is arranged to encode each of the frames at said frame size, and to transmit the encoded frames to a receiving terminal as part of a live video call.

2. The user terminal of claim 1 , wherein the pre-processing stage is configured to dynamically change the size of the image region in dependence on one or more channel conditions affecting said transmission, one or more processing resources of said user terminal, and/or one or more processing resources of the receiving terminal; and to adapt the border region so that each frame retains the frame size of the encoder.

3. The user terminal of claim 2 , wherein said adaptation change comprises varying the size of the image region between a plurality of different sizes smaller than the frame size of the encoder.

4. The user terminal of claim 3 , wherein said change comprises varying the size of the image region between the frame size of the encoder and at least one size smaller than the frame size of the encoder.

5. The user terminal of claim 2 , wherein the frame size of the encoder remains constant over a consecutive plurality of said frames while the size of the image region is changed.

6. The user terminal of claim 5 , wherein the encoder is operable to switch between different ones of a set of predetermined frame sizes.

7. The user terminal of claim 6 , wherein the encoder is configured to dynamically switch between different ones of the predetermined frame sizes, in dependence on one or more channel conditions affecting said transmission, one or more processing resources of said user terminal, and/or one or more processing resources of the receiving terminal.

8. The user terminal of claim 6 , wherein said change comprises varying the size of the image region in steps having a finer granularity than said predetermined frame sizes.

9. The user terminal of claim 1 , wherein the pre-processing stage is configured to perform said resizing by scaling down the source video image.

10. The user terminal of claim 1 , wherein the pre-processing stage is configured to embed alternative data in the border region which does not represent an image to be rendered at the receiving terminal.

11. The user terminal of claim 1 , wherein the border region has uniform colour and brightness within each of the modified frames, other than any modulation embedding non-image data.

12. The user terminal of claim 1 , wherein the border region is black.

13. The user terminal of claim 1 , wherein the border region remains constant over a plurality of the modified frames, other than any modulation embedding non-image data.

14. The user terminal of claim 1 , wherein the border region comprises no image content beyond that of the video image.

15. The user terminal of claim 1 , wherein the encoding of the border region comprises inter or intra frame prediction coding.

16. The user terminal of claim 1 , configured to signal information on said resizing to the receiving terminal, for use in scaling up the image region of the modified frames for display by the receiving terminal.

17. The user terminal of claim 16 , wherein the information on the resizing comprises one or more of: an indication of a percentage or fraction scaling, and/or a position of the image region within the modified frame.

18. A video telephony system comprising the user terminal of claim 16 and the receiving terminal, the receiving terminal comprising:
a decoder configured to decode each of the frames; and
a renderer configured to render the image portion of each decoded frame at said frame size;
wherein the renderer comprises a resizing stage configured to receive the information on the resizing performed by the pre-processing stage, and based on said information to scale up the image portion of each of the modified frames to said frame size, discarding the border region.

19. A user terminal for participating in video calls, comprising:
a decoder configured to receive and decode a sequence of frames from an encoder of a transmitting terminal, each frame having been encoded at a frame size of the encoder being a size in pixels and remaining constant while the resolution of the frames changes, and each frame comprising at least an image portion representing a source video image at a respective moment in time; and
a renderer configured to render the image portion of each decoded frame at the frame size of the encoder;
wherein at least some of the frames have been modified prior to encoding by the encoder, whereby the image region has been resized from the source video image to a size smaller than the frame size of the encoder, and combined with a border region such that the modified frame matches the frame size of the encoder; and
wherein the renderer comprises a resizing stage configured to receive information on the resizing, and based on said information to scale up the image portion of each of the modified frames to the frame size of the encoder, discarding the border region.

20. A computer-implemented method comprising:
supplying a sequence of outbound frames to a near-end encoder at a frame size of the near-end encoder, the frame size being a size in pixels at which the near-end encoder encodes frames of video, and each outbound frame comprising at least an image region representing a source video image at a respective moment in time; and
supplying at least some of said outbound frames to the near-end encoder in a modified form, by resizing the source video image to produce the image region of each modified frame with a size smaller than the frame size of the near-end encoder, and combining with a border region such that the modified frame matches the frame size of the near-end encoder, the resizing comprising cropping a portion of the source video image and replacing the cropped portion of the source video image with a portion of the border region;
encoding each of the frames at said frame size for transmission to a receiving terminal.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Carlsson Pontus
• Hemmendorff Magnus
• Hofbauer Konrad
• Nikiforov Sergei
• Zhao David Yuheng

Classifications

• H04N19/63

  using sub-band based transform, e.g. wavelets · CPC title

• H04N19/85

  using pre-processing or post-processing specially adapted for video compression · CPC title

• H04N19/167

  Position within a video image, e.g. region of interest [ROI] · CPC title

• H04N19/164

  Feedback from the receiver or from the transmission channel · CPC title

• H04N7/147 Primary

  Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q) · CPC title