Stereo image recording and playback

Patent metadata

Field	Value
Publication number	US-2017280133-A1
Application number	US-201415509038-A
Country	US
Kind code	A1
Filing date	Sep 9, 2014
Priority date	Sep 9, 2014
Publication date	Sep 28, 2017
Grant date	—

Abstract

The invention relates to forming a scene model and determining a first group of scene points, the first group of scene points being visible from a rendering viewpoint, determining a second group of scene points, the second group of scene points being at least partially obscured by the first group of scene points viewed from the rendering viewpoint, forming a first render layer using the first group of scene points and a second render layer using the second group of scene points, and providing the first and second render layers for rendering a stereo image. The invention also relates to receiving a first render layer and a second render layer comprising pixels, the first render layer comprising pixels corresponding to first parts of a scene viewed from a rendering viewpoint and the second render layer comprising pixels corresponding to second parts of the scene viewed from the rendering viewpoint, wherein the second parts of the scene are obscured by the first parts viewed from the rendering viewpoint, placing pixels of the first render layer and pixels of the second render layer in a rendering space,associating a depth value with the pixels, and rendering a stereo image using said pixels and said depth values.

First claim

1 - 68 . (canceled)

69 . A method, comprising:
forming a scene model using first image data from a first source image and second image data from a second source image, said scene model comprising scene points, each scene point having a location in a coordinate space of said scene; determining a first group of scene points, said first group of scene points being visible from a viewing point, said viewing point having a location in said coordinate space of said scene; determining a second group of scene points, said second group of scene points being at least partially obscured by said first group of scene points viewed from said viewing point; forming a first render layer using said first group of scene points and a second render layer using said second group of scene points, said first and second render layer comprising pixels; and providing said first and second render layers for rendering a stereo image.

70 . The method according to claim 69 , further comprising:
determining a third group of scene points, said third group of scene points being at least partially obstructed by said second group of scene points viewed from said viewing point; forming a third render layer using said third group of scene points, said third render layer comprising pixels; and providing said third render layer for rendering a stereo image.

71 . The method according to claim 69 , wherein said second render layer is a sparse layer comprising active pixels corresponding to scene points at least partially obstructed by said first group of scene points.

72 . The method according to claim 71 , further comprising:
forming dummy pixels in said second render layer, said dummy pixels not corresponding to scene points; and encoding said second render layer into a data structure using an image encoder.

73 . The method according to claim 69 , wherein forming said scene model comprises at least one of the following:
determining a three-dimensional location for said scene points by utilizing depth information for said source images; and using camera position of said source images and comparing image contents of said source images.

74 . The method according to claim 69 , further comprising:
forming one or more of said render layers to a two-dimensional image data structure, said image data structure comprising render layer pixels.

75 . The method according to claim 74 , wherein render layer pixels comprise color values and a transparency value such as an alpha value.

76 . The method according to claim 69 , further comprising:
forming data of at least two of said render layers into a collated image data structure, said collated image data structure comprising at least two segments, each segment corresponding to a respective render layer.

77 . A method comprising:
receiving a first render layer and a second render layer, said first and second render layer comprising pixels, said first render layer comprising pixels corresponding to first parts of a scene viewed from a rendering viewpoint and said second render layer comprising pixels corresponding to second parts of said scene viewed from said rendering viewpoint, wherein said second parts of said scene are obscured by said first parts viewed from said rendering viewpoint; placing pixels of said first render layer and pixels of said second render layer in a rendering space; associating depth values with said pixels; and rendering a left eye image and a right eye image using said pixels and said depth values.

78 . The method according to claim 77 , wherein said pixels of said first render layer and said second render layer comprise colour values and at least pixels of said first render layer comprise transparency values such as alpha values for rendering transparency of at least pixels of said first render layer.

79 . The method according to claim 77 , comprising:
determining whether a render layer to be rendered comprises semitransparent pixels; and in case said determining indicates a render layer comprises semitransparent pixels, enabling alpha blending in rendering of said render layer, otherwise disabling alpha blending in rendering said render layer.

80 . The method according to claim 77 , comprising:
receiving said first render layer and said second render layer from a data structure comprising pixel values as a two-dimensional image; and determining colour values for said pixels of said first and second render layers by using texture mapping.

81 . The method according to claim 77 , comprising:
receiving said first render layer and said second render layer from a data structure comprising pixel values as a two-dimensional image; and determining depth values for said pixels of said first and second render layers by using texture mapping, said depth values indicating a distance from a rendering viewpoint.

82 . The method according to claim 77 , comprising:
receiving said first render layer and said second render layer from a data structure comprising pixel values as a two-dimensional image; and determining viewing angle values for said pixels of said first and second render layers by using texture mapping.

83 . An apparatus comprising at least one processor, memory including computer program code, the memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following:
form a scene model using first image data from a first source image and second image data from a second source image, said scene model comprising scene points, each scene point having a location in a coordinate space of said scene; determine a first group of scene points, said first group of scene points being visible from a viewing point, said viewing point having a location in said coordinate space of said scene; determine a second group of scene points, said second group of scene points being at least partially obscured by said first group of scene points viewed from said viewing point; form a first render layer using said first group of scene points and a second render layer using said second group of scene points, said first and second render layer comprising pixels; and provide said first and second render layers for rendering a stereo image.

84 . The apparatus according to claim 83 , wherein the memory and the computer program code configured to, with the at least one processor, further cause the apparatus to perform at least the following:
determine a third group of scene points, said third group of scene points being at least partially obstructed by said second group of scene points viewed from said viewing point; form a third render layer using said third group of scene points, said third render layer comprising pixels; and provide said third render layer for rendering a stereo image.

85 . The apparatus according to claim 83 , wherein said second render layer is a sparse layer comprising active pixels corresponding to scene points at least partially obstructed by said first group of scene points.

86 . The apparatus according to claim 85 , wherein the memory and the computer program code configured to, with the at least one processor, further cause the apparatus to perform at least the following:
form dummy pixels in said second render layer, said dummy pixels not corresponding to scene points; and encode said second render layer into a data structure using an image encoder.

87 . The apparatus according to claim 83 , wherein forming said scene model comprises at least of the foll

Assignees

• Nokia Technologies Oy

Inventors

• Niemelä Marko
• Grönholm Kim
• Baldwin Andrew

Classifications

• H04N13/275 Primary

  from three-dimensional [3D] object models, e.g. computer-generated stereoscopic image signals · CPC title

• G06T15/04 Primary

  Texture mapping · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T7/50

  Depth or shape recovery · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title