Adapting Quantization

Patent metadata

Field	Value
Publication number	US-2016100166-A1
Application number	US-201414560669-A
Country	US
Kind code	A1
Filing date	Dec 4, 2014
Priority date	Oct 3, 2014
Publication date	Apr 7, 2016
Grant date	—

Abstract

A device comprising: an encoder for encoding a video signal representing a video image of a scene captured by a camera, and a controller. The encoder comprises a quantizer for performing a quantization on the video signal as part of said encoding. The controller is configured to receive skeletal tracking information from a skeletal tracking algorithm relating to one or more skeletal features of a user present in the scene, and based thereon to define one or more regions-of-interest within the video image corresponding to one or more bodily areas of the user, and to adapt the quantization to use a finer quantization granularity within the one or more regions-of-interest than outside the one or more regions-of interest.

First claim

1 . A device comprising:
an encoder for encoding a video signal representing a video image of a scene captured by a camera, the encoder comprising a quantizer for performing a quantization on said video signal as part of said encoding; and a controller configured to receive skeletal tracking information from a skeletal tracking algorithm relating to one or more skeletal features of a user present in said scene, and based thereon to define one or more regions-of-interest within the video image corresponding to one or more bodily areas of the user, and to adapt the quantization to use a finer quantization granularity within the one or more regions-of-interest than outside the one or more regions-of interest.

2 . The device of claim 1 , wherein the controller is configured to define a plurality of different regions-of-interest each corresponding to a respective bodily area of the user, and to adapt the quantization to use a finer quantization granularity within each of said plurality of regions-of-interest than outside the plurality of regions-of-interest.

3 . The device of claim 2 , wherein one or more of the different regions-of-interest are quantized with the finer quantization granularity only at some times and not others.

4 . The device of claim 3 , wherein the controller is configured to adaptively select which of the different regions-of-interest is currently quantized with the finer quantization granularity in dependence on a current bitrate constraint.

5 . The device of claim 4 , wherein the bodily areas are assigned an order of priority, and the controller is configured to perform said selection according to the order of priority of the bodily areas to which the different regions-of-interest correspond.

6 . The device of claim 2 , wherein the controller is configured to adapt the quantization to use different levels of quantization granularity within different ones of said plurality of regions-of interest, each being finer than outside the plurality of regions-of-interest.

7 . The device of claim 6 , wherein said bodily areas are assigned an order of priority, and the controller is configured to set the different levels according to the order of priority of the bodily areas to which the different regions-of-interest correspond.

8 . The device of claim 1 , wherein each of the bodily areas is one of:
(a) the user's whole body; (b) the user's head, torso and arms; (c) the user's head, thorax and arms; (d) the user's head and shoulders; (e) the user's head; (f) the user's torso (g) the user's thorax; (h) the user's abdomen; (i) the user's arms and hands; (j) the user's shoulders; or (k) the user's hands.

9 . The device of claim 5 , wherein the order of priority is:
(i) the user's head; (ii) the user's head and shoulders; or head, thorax and arms; or head, torso and arms; (iii) the user's whole body; such that (iii) is quantized with the finer quantization if the bitrate constraint allows, and if not only (ii) is quantized with the finer quantization if the bitrate constraint allows, and if not only (i) is quantized with the finer quantization.

10 . The device of claim 7 , wherein the order of priority is:
(i) the user's head; (ii) the user's hands, arm, shoulders, thorax and/or torso; (iii) the rest of the user's whole body; such that is (i) quantized with a first level of quantization granularity, (ii) is quantized with one or more second levels of quantization granularity, and (iii) is quantized with a third level of quantization granularity, the first level being finer than each of the one or more second levels, each of the second levels being finer than the third level, and the third level being finer than outside the regions-of-interest.

11 . The device of claim 1 , comprising a transmitter configured to transmit the encoded video signal over a channel to at least one other device.

12 . The device of claim 4 , comprising a transmitter configured to transmit the encoded video signal over a channel to at least one other device, wherein the controller is configured to determine an available bandwidth of said channel, and said bitrate constraint is equal to or otherwise limited by the available bandwidth.

13 . The device of claim 1 , wherein the controller is configured to apply a successive increase in the coarseness of the quantization granularity from at least one of the one or more regions-of-interest toward the outside.

14 . The device of claim 1 , wherein the controller is configured to apply a spring model to smooth a motion of the one or more regions-of-interest as they follow the one or more corresponding bodily areas based on the skeletal tracking information.

15 . The device of claim 1 , comprising a transmitter for transmitting the encoded video signal over a network.

16 . The device of claim 1 , wherein the skeletal tracking algorithm is implemented on said device and is configured to determine said skeletal tracking information based on one or more separate sensors other than said camera.

17 . The device of claim 1 , comprising dedicated graphics processing resources and general purpose processing resources, wherein the skeletal tracking algorithm is implemented in the dedicated graphics processing resources and the encoder is implemented in the general purpose processing resources.

18 . The device of claim 17 , wherein the general purpose processing resources comprise a general purpose processor and the dedicated graphics processing resources comprise a separate graphics processor, the encoder being implemented in the form of code arranged to run on the general purpose processor and the skeletal tracking algorithm being implemented in the form of code arranged to run on the graphics processor.

19 . A computer program product comprising code embodied on a computer-readable storage medium and configured so as when run on one or more processors to perform operations of:
encoding a video signal representing a video image of a scene captured by a camera, the encoding comprising performing a quantization on said video signal; receiving skeletal tracking information from a skeletal tracking algorithm, relating to one or more skeletal features of a user present in said scene; based on the skeletal tracking information, defining one or more regions-of-interest within the video image corresponding to one or more bodily areas of the user; and adapting the quantization to use a finer quantization granularity within the one or more regions-of-interest than outside the one or more regions-of interest.

20 . A method comprising:
encoding a video signal representing a video image of a scene captured by a camera, the encoding comprising performing a quantization on said video signal; receiving skeletal tracking information from a skeletal tracking algorithm, relating to one or more skeletal features of a user present in said scene; based on the skeletal tracking information, defining one or more regions-of-interest within the video image corresponding to one or more bodily areas of the user; and adapting the quantization to use a finer quantization granularity within the one or more regions-of-interest than outside the one or more regions-of interest.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Dragne Lucian
• Hess Hans Peter

Classifications

• G06K2009/4666

  Physics · mapped topic

• G06K9/00375

  Physics · mapped topic

• G06K9/46

  Physics · mapped topic

• H04N19/124 Primary

  Quantisation · CPC title

• H04N19/51

  Motion estimation or motion compensation · CPC title