Efficient encoding of multiple views

Patent metadata

Field	Value
Publication number	US-9986258-B2
Application number	US-29451507-A
Country	US
Kind code	B2
Filing date	Mar 23, 2007
Priority date	Mar 31, 2006
Publication date	May 29, 2018
Grant date	May 29, 2018

Abstract

A new method of encoding multiple view image information into an image signal including adding to the image signal a first image of pixel values representing one or more objects captured by a first camera; adding to the image signal a map comprising for respective sets of pixels of the first image respective values, representing a three-dimensional position in space of a region of the one or more objects represented by the respective set of pixels, and adding to the image signal a partial representation of a second image of pixel values representing one or more objects captured by a second camera, the partial representation including at least information of the majority of the pixels representing regions of the one or more objects not visible to the first camera.

First claim

The invention claimed is:

1. A non-transitory memory medium having processor readable program code for operating on a processor for performing a method of encoding multiple view image information into an image signal, the method comprising acts of:
the processor
adding to the image signal a first image of pixel values representing one or more objects captured by a first camera;
adding to the image signal a map comprising for respective sets of pixels of the first image respective values, representing a three-dimensional position in space of a region of the one or more objects represented by the respective set of pixels;
adding to the image signal a partial representation of a second image of pixel values representing one or more objects captured by a second camera, the partial representation mostly comprising at least information of a substantial majority of the pixels representing regions of the one or more objects not visible to the first camera, the partial representation not including a substantial majority of portions of the second image visible to the first camera; and
transmitting the image signal to at least one of a storage device and a receiver, wherein the transmitted image signal includes the first image of pixel values, the map and the partial representation of the second image of pixel values.

2. The medium as claimed in claim 1 , the method further comprising an act of adding to the image signal a second map comprising at least a partial representation of, for respective sets of pixels of the second image, respective values representing a three-dimensional position in space of a region of the one or more objects represented by the respective set of pixels.

3. The medium as claimed in claim 1 , the method further comprising an act of automatically deriving by means of a disparity calculation algorithm for sets of pixels in the first image disparity vectors pointing to corresponding sets of pixels in the second image, and the act of adding to the image signal a map, consisting of adding the map comprising the disparity vectors.

4. The medium as claimed in claim 1 , the method further comprising an act of obtaining from a range determination device respective ranges for sets of pixels in the first image, and the act of adding to the image signal a map comprising an act of adding the map including the ranges.

5. The medium as claimed in claim 1 , the method further comprising an act of adjusting manually the values of the map prior to the adding to the image signal.

6. The medium as claimed in claim 1 , the method further comprising an act of adjusting manually the partial representation prior to the adding to the image signal.

7. The medium as claimed in claim 2 , the method further comprising an act of adding to the image signal an indication of positions in the map indicating a boundary between a first object and a second object.

8. The medium as claimed in claim 1 , wherein the act of adding to the image signal the partial representation of the second image comprises acts of specifying and adding to the image signal a representation of an encompassing shape encompassing the region of pixels in the partial representation of the second image.

9. The medium as claimed in claim 1 , wherein the act of adding to the image signal the partial representation of the second image comprises an act of specifying regions of the second image which are not encoded in the partial representation.

10. The medium as claimed in claim 1 , the method further comprising acts of:
performing a morphological analysis on regions comprised in the partial representation of the second image, and
modifying the partial representation based on the morphological analysis prior to adding the partial representation to the image signal, the morphological analysis comprising an act of making a determination of the largest width of the respective regions.

11. An apparatus for generating an encoding into an image signal of multiple view image information, the apparatus comprising:
a processor; and
a memory with instructions executable by the processor;
the instructions when executed by the processor performing acts of:
adding to the image signal a first image of pixel values representing one or more objects captured by a first camera;
adding to the image signal a map comprising for respective sets of pixels of the first image respective values representing a three-dimensional position in space of a region of the one or more objects represented by the respective set of pixels;
adding to the image signal a partial representation of a second image of pixel values representing one or more objects captured by a second camera, the partial representation mostly comprising at least information of a substantial majority of the pixels representing regions of the one or more objects not visible to the first camera, the partial representation not including a substantial majority of portions of the second image visible to the first camera; and
transmitting the image signal to at least one of a storage device and a receiver, wherein the transmitted image signal includes the first image of pixel values, the map and the partial representation of the second image of pixel values.

12. The apparatus as claimed in claim 11 , the instructions when executed by the processor performing a further act of adding to the image signal an indication of positions in the map indicating a boundary between a first object and a second object.

13. The apparatus as claimed in claim 11 , the instructions when executed by the processor performing a further act of providing a user interface unit arranged to give a human operator control over at least one of the contents of the map and the partial representation.

14. An image signal receiver, comprising:
a processor; and
a memory with instructions executable by the processor;
the instructions when executed by the processor performing acts of:
extracting from an image signal a first image of pixel values representing one or more objects captured by a first camera;
extracting from the image signal a map comprising for respective sets of pixels of the first image respective values representing a three-dimensional position in space of a region of the one or more objects represented by the respective set of pixels;
extracting from the image signal a partial representation of a second image of pixel values representing one or more objects captured by a second camera, the partial representation mostly comprising at least information of a substantial majority of the pixels representing regions of the one or more objects not visible to the first camera, the partial representation not including a substantial majority of portions of the second image visible to the first camera; and
transmitting the image signal to at least one of a storage device and an image rendering device, wherein the transmitted image signal includes the first image of pixel values, the map and the partial representation of the second image of pixel values.

15. A display capable of generating at least two views of an image comprising:
the image signal receiver as claimed in claim 14 , comprising:
a display configured to generate at least two views of an image,
the instructions when executed by the processor performing further acts of:
generating two images from the image signal data received by the image signal receiver; and
generating from the two images further images of a format appropriate for the display.

16. A non-transitory memory medium having processor readable program code for operating on a processor for performi

Assignees

• Bruls Wilhelmus Hendrikus Alfonsus
• Varekamp Christiaan
• Braspenning Ralph
• Koninklijke Philips Nv

Inventors

• Bruls Wilhelmus Hendrikus Alfonsus
• Varekamp Christiaan
• Braspenning Ralph

Classifications

• H04N2213/005

  Aspects relating to the "3D+depth" image format · CPC title

• H04N13/0022

  Electricity · mapped topic

• H04N19/00

  Methods or arrangements for coding, decoding, compressing or decompressing digital video signals · CPC title

• H04N19/597 Primary

  specially adapted for multi-view video sequence encoding · CPC title

• H04N13/00

  Stereoscopic video systems; Multi-view video systems; Details thereof · CPC title