Image encoding and display

Patent metadata

Field	Value
Publication number	US-10306202-B2
Application number	US-201515112324-A
Country	US
Kind code	B2
Filing date	Feb 25, 2015
Priority date	Feb 26, 2014
Publication date	May 28, 2019
Grant date	May 28, 2019

Abstract

Aspects of the technology encompass image encoding, which includes generating image content according to a viewpoint defined by image viewpoint data. Successive output images are generated such that each output image includes image content generated according to one or more viewpoints, and encoding metadata associated with each output image which indicates each viewpoint relating to image content contained in that output image, and which defines which portions of that output image were generated according to each of those viewpoints. An image display method for generating successive display images from successive input images is also provided, which includes re-projecting portions of each input image to form a respective display image, according to any differences between a desired display viewpoint and a particular viewpoint defined for that portion by metadata associated with a respective input image. The metadata indicating viewpoints relating to image content contained in the respective input images.

First claim

The invention claimed is:

1. An image display method for generating successive display images from successive input images each including image content generated according to one or more viewpoints, the input images each having associated metadata which indicates each viewpoint relating to image content contained in that input image, and which defines which portions of that input image were generated according to each respective viewpoint, the method comprising:
re-projecting, by one or more processors, the portions of each input image to form a respective display image, by image rotation and scaling to correct for any differences between a desired display viewpoint and a particular viewpoint defined for that portion by the metadata associated with a respective input image,
in which the input images have associated depth data indicating the image depth of one or more image features, and re-projecting comprises repositioning one or more image features within the re-projected image according to the depth data.

2. A method according to claim 1 , further comprising:
detecting a current position and/or current orientation of a viewer's head; and
deriving the desired display viewpoint in dependence upon the detected current position and/or current orientation of the user's head.

3. A method according to claim 1 , further comprising:
detecting an initial position and/or orientation of the viewer's head;
providing the detected initial position and/or initial orientation to an image generator to generate image content according to that initial position and/or initial orientation as a viewpoint.

4. A method according to claim 1 , further comprising displaying the re-projected image using a display.

5. A method according to claim 4 , in which the display is a head-mountable display and the position and/or orientation of the viewer's head is detected by detecting a position and/or orientation of the head-mountable display.

6. A method according to claim 1 , in which the step of re-projecting comprises detecting an overlapping portion between a given generated image and a required re-projected image, and reproducing the overlapping portion as part of the re-projected image.

7. A method according to claim 6 , comprising filling portions of the re-projected image other than the overlapping portion with image content from a selected image source.

8. A method according to claim 1 , in which the metadata is provided as pixel data within the input images, the method further comprising:
omitting the pixel data from the display images.

9. An image display system operable to generate successive display images from successive input images that each include image content generated according to one or more viewpoints, the input images each having associated metadata which indicates each viewpoint relating to image content contained in that input image, and which defines which portions of that input image were generated according to each respective viewpoint, the system comprising:
a display element observable by a viewer; and
an image re-projector configured to re-project the portions of each input image to form a respective display image, by image rotation and scaling to correct for any differences between a desired display viewpoint and a viewpoint defined for that portion by the metadata associated with a respective input image,
in which the input images have associated depth data indicating the image depth of one or more image features, and re-projecting comprises repositioning one or more image features within the re-projected image according to the depth data.

10. A system according to claim 9 , the system being a head-mountable display system.

Assignees

• Sony Computer Entertainment Europe Ltd
• Sony Interactive Entertainment Europe Ltd

Inventors

• Raghoebardajal Sharwin Winesh
• Bickerstaff Ian Henry

Classifications

• H04N21/84 Primary

  Generation or processing of descriptive data, e.g. content descriptors {(systems specially adapted for using meta-information in broadcast systems H04H60/73)} · CPC title

• A63F2300/8082

  Virtual reality · CPC title

• H04N2213/005

  Aspects relating to the "3D+depth" image format · CPC title

• H04N13/128

  Adjusting depth or disparity · CPC title

• H04N13/279

  the virtual viewpoint locations being selected by the viewers or determined by tracking · CPC title