Adapting Encoding Properties

Patent metadata

Field	Value
Publication number	US-2016100165-A1
Application number	US-201414559617-A
Country	US
Kind code	A1
Filing date	Dec 3, 2014
Priority date	Oct 3, 2014
Publication date	Apr 7, 2016
Grant date	—

Abstract

A device, computer program and method for encoding a video signal representing a video image of a scene captured by a camera. The device comprises a controller for receiving skeletal tracking information from a skeletal tracking algorithm relating to one or more skeletal features of a user when present in the scene, wherein the controller is configured to adapt a current value of one or more motion-related properties of the encoding in dependence on the skeletal tracking information as currently relating to the scene.

First claim

1 . A device comprising:
an encoder for encoding a video signal representing a video image of a scene captured by a camera; and a controller for receiving skeletal tracking information from a skeletal tracking algorithm relating to one or more skeletal features of a user when present in said scene, wherein the controller is configured to adapt a current value of one or more motion-related properties of the encoding in dependence on the skeletal tracking information as currently relating to said scene.

2 . The device of claim 1 , wherein the encoder comprises a quantizer for performing a quantization on said video signal as part of said encoding, and said one or more properties comprise a granularity of the quantization.

3 . The device of claim 1 , wherein said one or more properties comprise a frame rate of the encoding.

4 . The device of claim 2 , wherein said properties comprise both the quantization granularity and a frame rate of the encoding.

5 . The device of claim 1 , wherein the controller is configured to perform said adaptation of the one or more properties such that a bitrate of the encoding remains within a current bitrate budget.

6 . The device of claim 5 , wherein the controller is configured to perform said adaptation such that the bitrate of the encoding remains constant at the current bitrate budget.

7 . The device of claim 1 , wherein said adaptation comprises adapting the one or more properties in dependence on whether or not a user is currently detected to be present in said scene based on the skeletal tracking information.

8 . The device of claim 1 , wherein said adaptation comprises applying a finer granularity quantization and/or lower frame rate when no user is currently detected to be present in the scene based on the skeletal tracking information, and a coarser granularity quantization and/or higher frame rate when a user is currently detected to be present in said scene based on the skeletal tracking information.

9 . The device of claim 1 , wherein said adaptation comprises adapting the one or more properties in dependence on motion of the user relative to said scene, as currently detected based on the skeletal tracking information.

10 . The device of claim 9 , wherein said adaptation comprises adapting the one or more properties in dependence on whether or not a user is detected to be moving relative to said scene based on the skeletal tracking information.

11 . The device of claim 1 , wherein said adaptation comprises applying a finer granularity quantization and/or lower frame rate when no user is currently detected to be moving in the scene based on the skeletal tracking information, and a coarser granularity quantization and/or higher frame rate when a user is currently detected to be moving in the scene based on the skeletal tracking information.

12 . The device of claim 9 , wherein said adaptation comprises adapting said one or more properties in dependence on a degree of motion of the user currently detected based on the skeletal tracking information.

13 . The device of claim 4 , wherein said adaptation comprises:
(i) applying a finer granularity quantization and lower frame rate when no user is currently detected to be present in the scene based on the skeletal tracking information, (ii) applying an intermediate granularity quantization and frame rate when a user is detected based on the skeletal tracking information to be present in the scene but not moving, and (iii) applying a coarser granularity quantization and higher frame rate when a user is detected based on the skeletal tracking information to be both present in the scene and moving.

14 . The device of claim 8 , wherein the controller is configured to trigger inclusion of a new key frame in the encoding when switching between the different quantization granularities.

15 . The device of claim 1 , comprising a transmitter for transmitting the encoded video signal over a network.

16 . The device of claim 1 , wherein the skeletal tracking algorithm is implemented on said device and is configured to determine said skeletal tracking information based on one or more separate sensors other than said camera.

17 . The device of claim 1 , comprising dedicated graphics processing resources and general purpose processing resources, wherein the skeletal tracking algorithm is implemented in the dedicated graphics processing resources and the encoder is implemented in the general purpose processing resources.

18 . The device of claim 17 , wherein the general purpose processing resources comprise a general purpose processor and the dedicated graphics processing resources comprise a separate graphics processor, the encoder being implemented in the form of code arranged to run on the general purpose processor and the skeletal tracking algorithm being implemented in the form of code arranged to run on the graphics processor.

19 . A computer program product comprising code embodied on a computer-readable storage medium and configured so as when run on one or more processors to perform operations of:
encoding a video signal representing a video image of a scene captured by a camera; receiving skeletal tracking information from a skeletal tracking algorithm, the skeletal tracking information relating to one or more skeletal features of a user when present in said scene; and adapting a current value of one or more motion-related properties of the encoding in dependence on the skeletal tracking information as currently relating to said scene.

20 . A method comprising:
encoding a video signal representing a video image of a scene captured by a camera; receiving skeletal tracking information from a skeletal tracking algorithm, the skeletal tracking information relating to one or more skeletal features of a user when present in said scene; and adapting a current value of one or more motion-related properties of the encoding in dependence on the skeletal tracking information as currently relating to said scene.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Zhao David Yuheng

Classifications

• H04N19/137 Primary

  Motion inside a coding unit, e.g. average field, frame or block difference · CPC title

• H04N19/115

  Selection of the code volume for a coding unit prior to coding · CPC title

• H04N19/146

  Data rate or code amount at the encoder output · CPC title

• H04N19/124 Primary

  Quantisation · CPC title

• H04N19/132

  Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking · CPC title