Technique for view synthesis

Patent metadata

Field	Value
Publication number	US-9609210-B2
Application number	US-201414511481-A
Country	US
Kind code	B2
Filing date	Oct 10, 2014
Priority date	Oct 11, 2013
Publication date	Mar 28, 2017
Grant date	Mar 28, 2017

Abstract

A technique for computing an image of a virtual view based on a plurality of camera views is presented. One or more cameras provide the plurality of camera views. As to a method aspect of the technique, two or three camera views that at least partially overlap and that at least partially overlap with the virtual view are selected among the plurality of camera views. The image of the virtual view is computed based on objects in the selected camera views using a multilinear relation that relates the selected camera views and the virtual view.

First claim

The invention claimed is:

1. A method, performed by a device, for computing an image of a virtual view, the method comprising:
receiving a plurality of camera views from one or more cameras;
selecting from among the plurality of camera views two or three camera views that at least partially overlap and that at least partially overlap with the virtual view; and
computing the image of the virtual view based on objects in the selected camera views using a multilinear relation that relates the selected camera views and the virtual view, wherein
the multilinear relation is represented by a trifocal tensor (T), if two camera views are selected, and wherein the multilinear relation is represented by a quadrifocal tensor (Q), if three camera views are selected, wherein
the virtual view is specified by parameters of a projection, the parameters of the projection specifying the virtual view include a zoom level of the virtual view, wherein
a mesh is represented in memory, the mesh including a vertex for each of the camera views and an edge for each pair of overlapping camera views among the plurality of camera views, and
the selection of the camera views from among the plurality of camera views depends on the zoom level of the virtual view and one or more other parameters of the projection specifying the virtual view.

2. The method of claim 1 , wherein different parts of the image of the virtual view are computed based on different selections of two or three camera views.

3. The method of claim 2 , wherein the different parts form a partition that is indicative of the two or three at least partially overlapping camera views among the plurality of camera views for each of the parts.

4. The method of claim 1 , wherein parameters of a projection are specified for each of the camera views.

5. The method of claim 1 , wherein the mesh indicates pairs and triples of overlapping camera views as a function of the projection parameters of the virtual view.

6. The method of claim 1 , further comprising computing the multilinear relation as a function of one or more of the projection parameters of the virtual view.

7. The method of claim 6 , wherein a projection matrix (P) is specified for each of the selected camera views and the virtual view, and wherein the multilinear relation is computed based on the projection matrices.

8. The method of claim 1 , wherein the parameters of the projection specifying the virtual view further specify at least one of: a position of a projection center of the virtual view; an orientation of an optical axis; and/or and a projection plane of the virtual view.

9. The method of claim 1 , wherein a sequence of images of the virtual view is computed based on changing projection parameters of the virtual view.

10. A computer program product comprising a non-transitory computer readable medium comprising program code for carrying out the method according to claim 1 when the computer program product is executed on one or more computing devices.

11. A device for computing an image of a virtual view based on a plurality of camera views, the device comprising:
a memory; and
a processor, wherein the processor is configured to:
receive a plurality of camera views from one or more cameras;
select from among the plurality of camera views two or three camera views that at least partially overlap and that at least partially overlap with the virtual view; and
compute the image of the virtual view based on objects in the selected camera views using a multilinear relation that relates the selected camera views and the virtual view, wherein
the multilinear relation is represented by a trifocal tensor (T), if two camera views are selected, and wherein the multilinear relation is represented by a quadrifocal tensor (Q), if three camera views are selected, wherein
the virtual view is specified by parameters of a projection, and
the parameters of the projection specifying the virtual view include a zoom level of the virtual view, wherein
a mesh is represented in memory, the mesh including a vertex for each of the camera views and an edge for each pair of overlapping camera views among the plurality of camera views, and
the selection of the camera views from among the plurality of camera views depends on the zoom level of the virtual view and one or more other parameters of the projection specifying the virtual view.

12. The device of claim 11 , further comprising a receiver adapted to wirelessly receive image data from the one or more cameras in hardware.

Assignees

• Ericsson Telefon Ab L M
• ERICSSON TELEFON AB L M (publ)

Inventors

• Djordjevic Branko
• Cagman Can
• Langer Alexander

Classifications

• H04N23/698 Primary

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• G06T15/205 Primary

  Image-based rendering · CPC title

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• H04N23/45

  for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images · CPC title

• H04N5/2258

  Electricity · mapped topic