View synthesis distortion model for multiview depth video coding

Patent metadata

Field	Value
Publication number	US-9307252-B2
Application number	US-201213488127-A
Country	US
Kind code	B2
Filing date	Jun 4, 2012
Priority date	Jun 4, 2012
Publication date	Apr 5, 2016
Grant date	Apr 5, 2016

Abstract

The disclosed subject matter relates to providing a view synthesis distortion model (VSDM) for multiview video coding (MVC). The disclosed VSDM can facilitate determining quantization values and rate values based on model parameters for encoding depth information. Further, the VSDM can facilitate compression of depth information based on the determined quantization values and rate values. Compression of depth information can provide for reduces bandwidth consumption for dissemination of encoded multiview content for applications such as 3D video, freepoint TV, etc. Further, a feedback element can be employed to update the VSDM based on a comparison of a reconstituted version of the content, from coded depth information, against reference version of the content, from reference depth information.

First claim

What is claimed is:

1. A system, comprising:
at least one memory that stores computer-executable instructions; and
at least one processor, communicatively coupled to the at least one memory, that facilitates execution of the computer-executable instructions to at least:
receive an encoder configured to encode a first portion of a multiview depth information input stream associated with depth information for a smooth region of a scene differently than a second portion of the multiview depth information input stream associated with other depth information for a texture region of the scene;
receive a view synthesis distortion model comprising a first slope region corresponding to the depth information for the smooth region and a second slope region corresponding to the other depth information for the texture region, a first slope value of the first slope region is updated based on pixel distortion related to perturbing pixel position in the smooth region, and a second slope value of the second slope region is updated based on pixel distortion related to perturbing pixel position in the texture region;
encode the first portion of the multiview depth information input stream based on the encoder and the first slope value; and
encode the second portion of the multiview depth information input stream based on the encoder and the second slope value, different from the first slope value.

2. The system of claim 1 , wherein the encoder is a moving picture experts group (MPEG) codec.

3. The system of claim 2 , wherein the encoder is an H.264/advanced video coding (AVC) codec.

4. The system of claim 1 , wherein the multiview depth information input stream includes a plurality of contemporaneous disparate views of a scene facilitating a determination of depth information for an object in the scene.

5. The system of claim 1 , wherein the multiview depth information input stream includes predetermined depth information about an object in the scene and wherein the view synthesis distortion model comprises a parameter determined from intermittently injecting a determined level of noise into the multiview depth information input stream.

6. The system of claim 1 , wherein the view synthesis distortion model includes a slope parameter related to a texture complexity of the scene.

7. The system of claim 6 , wherein the slope parameter correlates a rendered view error to a depth information error with the texture complexity of the scene.

8. The system of claim 7 , wherein the depth information error results, at least in part, from an error in a quantization of depth information.

9. The system of claim 1 , wherein the view synthesis distortion model includes a residual parameter that correlates an average mean absolute difference of a residual between a reference signal and a predictive signal.

10. The system of claim 1 , wherein the at least one processor further facilitates the execution of the computer-executable instructions to determine a quantization parameter based on a slope value of the view synthesis distortion model.

11. The system of claim 1 , wherein the at least one processor further facilitates the execution of the computer-executable instructions to determine a bit rate value based, at least in part, on a rate-distortion model employing reconstructed depth information.

12. The system of claim 1 , wherein the at least one processor further facilitates execution of the computer-executable instructions to at least:
decode at least a portion of the encoded first portion of the multiview depth information input stream to generate at least a decoded first portion;
render at least the decoded first portion of the multiview depth information input stream as a current render; and
update the view synthesis distortion model based, at least in part, upon a comparison of the current render with a reference render.

13. A method, comprising:
receiving, by a system including a processor, an encoder facilitating encoding of a smooth region of a scene comprised in a multiview input stream and differently encoding of a texture region of the scene; and
encoding, by the system, the multiview input stream based, at least in part, on a first slope value determined from a view synthesis distortion model and the encoder, wherein the view synthesis distortion model comprises distortion values corresponding to shifts in positions for pixels of the smooth region and other distortion values corresponding to other shifts in positions for pixels of the texture region to facilitate determining the first slope value for the smooth region different from a second slope value for the texture region.

14. The method of claim 13 , wherein receiving the encoder includes receiving a moving picture experts group (MPEG) codec.

15. The method of claim 14 , wherein receiving the encoder includes receiving a H.264/ advanced video coding (AVC) codec.

16. The method of claim 13 , further comprising:
receiving, by the system, the multiview input stream including receiving a plurality of contemporaneous disparate views of a scene facilitating determination of depth information for an object in the scene.

17. The method of claim 13 , further comprising:
receiving, by the system, a multiview input stream including receiving predetermined depth information for an object in the scene.

18. The method of claim 13 , further comprising:
receiving, by the system, the view synthesis distortion model including receiving a slope parameter.

19. The method of claim 13 , further comprising:
receiving, by the system, the view synthesis distortion model including receiving a residual parameter that correlates an average mean absolute difference of a residual between a reference signal and a predictive signal.

20. The method of claim 13 , further comprising:
determining, by the system, a quantization value based on a slope parameter of the view synthesis distortion model.

21. The method of claim 13 , further comprising:
determining, by the system, a bit rate value based, at least in part, on a rate-distortion model employing reconstructed depth information.

22. The method of claim 13 , further comprising:
decoding, by the system, the encoded multiview input stream;
rendering, by the system, the decoded multiview input stream into a version;
analyzing, by the system, the version relative to a reference version; and
updating, by the system, the view synthesis distortion model based, at least in part, upon the analyzing.

23. A computer-readable storage device having executable instructions that, in response to execution, cause a device including a processor to perform operations, comprising:
receiving an encoder facilitating encoding of a smooth region of a scene corresponding to a first portion of a multiview depth input stream and differently encoding a textured region of the scene corresponding to a second portion of the multiview depth input stream; and
encoding the first portion of the multiview depth input stream based at least in part on a first portion of a view synthesis distortion model and the encoder, encoding the second portion of the multiview depth input stream based at least in part on a second portion of the view synthesis distortion model and the encoder, wherein the first portion of the view synthesis distortion model comprises a parameter determined from pixel distortion related to perturbing pixel position in the smooth region, and the second portion of the view synthesis distortion mo

Assignees

• Zhang Yun
• Kwong Sam Tak Wu
• Univ City Hong Kong

Inventors

• Zhang Yun
• Kwong Sam Tak Wu

Classifications

• H04N13/161

  Encoding, multiplexing or demultiplexing different image signal components (for multi-view video sequence encoding H04N19/597) · CPC title

• H04N19/124

  Quantisation · CPC title

• H04N19/172

  the region being a picture, frame or field · CPC title

• H04N19/597

  specially adapted for multi-view video sequence encoding · CPC title

• H04N19/115

  Selection of the code volume for a coding unit prior to coding · CPC title