Multi-pass add-on tool for coherent and complete view synthesis

Patent metadata

Field	Value
Publication number	US-11722653-B2
Application number	US-202117217588-A
Country	US
Kind code	B2
Filing date	Mar 30, 2021
Priority date	Mar 19, 2019
Publication date	Aug 8, 2023
Grant date	Aug 8, 2023

Abstract

An embodiment of an image processor for immersive video includes technology to re-order patches from a plurality of views based on one or more of relative position and orientation related information for a desired synthesized view, select a set of views to be used in each view synthesis pass, perform two or more view synthesis passes for the synthesized view to provide two or more intermediate view synthesis results, and mask and merge the two or more intermediate view synthesis results to provide a final view synthesis result. Other embodiments are disclosed and claimed.

First claim

What is claimed is:

1. An apparatus to merge camera views, comprising:
at least one memory;
machine readable instructions; and
processor circuitry to execute the machine readable instructions to at least:
order a plurality of views based on (a) a target synthesized view and (b) parameters corresponding to the target synthesized view;
generate a first intermediate view synthesis result corresponding to a first one of the ordered plurality of views;
generate a second intermediate view synthesis result corresponding to a second one of the ordered plurality of views;
mask the first intermediate view synthesis result with a first mask; and
generate a final view synthesis result by merging the second intermediate view synthesis result with an output of the first mask.

2. The apparatus as defined in claim 1 , wherein the parameters include distance information, the processor circuitry to arrange the order of the plurality of views based on a distance between respective ones of the plurality of views.

3. The apparatus as defined in claim 1 , wherein the parameters of the target synthesized view include (a) position information and (b) orientation information.

4. The apparatus as defined in claim 1 , wherein the processor circuitry is to identify a degree of overlap between (a) the first one of the ordered plurality of views and the target synthesized view and (b) the second one of the ordered plurality of views and the target synthesized view.

5. The apparatus as defined in claim 4 , wherein the processor circuitry is to order the plurality of views based on one of the first one of the ordered plurality of views or the second one of the ordered plurality of views having a greater one of the degree of overlap.

6. The apparatus as defined in claim 1 , wherein the processor circuitry is to generate additional intermediate view synthesis results based on at least one of (a) a number of cameras corresponding to the plurality of views or (b) a number of positions of the cameras.

7. The apparatus as defined in claim 1 , wherein the plurality of views includes at least one of orthogonal position information or yaw-pitch-roll orientation information.

8. At least one machine readable storage device or storage disk comprising instructions that, when executed by processor circuitry, cause the processor circuitry to at least:
order a plurality of views based on (a) a target synthesized view and (b) parameters corresponding to the target synthesized view;
generate a first intermediate view synthesis result corresponding to a first one of the ordered plurality of views;
generate a second intermediate view synthesis result corresponding to a second one of the ordered plurality of views;
mask the first intermediate view synthesis result with a first mask; and
generate a final view synthesis result by merging the second intermediate view synthesis result with an output of the first mask.

9. The machine readable storage device or storage disk as defined in claim 8 , wherein the instructions, when executed, cause the processor circuitry to arrange the order of the plurality of views based on a distance between respective ones of the plurality of views.

10. The machine readable storage device or storage disk as defined in claim 8 , wherein the instructions, when executed, cause the processor circuitry to identify the parameters of the target synthesized view as at least one of (a) position information and (b) orientation information.

11. The machine readable storage device or storage disk as defined in claim 8 , wherein the instructions, when executed, cause the processor circuitry to identify a degree of overlap between (a) the first one of the ordered plurality of views and the target synthesized view and (b) the second one of the ordered plurality of views and the target synthesized view.

12. The machine readable storage device or storage disk as defined in claim 11 , wherein the instructions, when executed, cause the processor circuitry to order the plurality of views based on one of the first one of the ordered plurality of views or the second one of the ordered plurality of views having a greater one of the degree of overlap.

13. The machine readable storage device or storage disk as defined in claim 8 , wherein the instructions, when executed, cause the processor circuitry to generate additional intermediate view synthesis results based on at least one of (a) a number of cameras corresponding to the plurality of views or (b) a number of positions of the cameras.

14. The machine readable storage device or storage disk as defined in claim 8 , wherein the instructions, when executed, cause the processor circuitry to identify at least one of orthogonal position information or yaw-pitch-roll orientation information in the plurality of views.

15. An apparatus to merge camera views, comprising:
views reordering circuitry to order a plurality of views based on (a) a target synthesized view and (b) parameters corresponding to the target synthesized view;
multi-pass view circuitry to:
generate a first intermediate view synthesis result corresponding to a first one of the ordered plurality of views; and
generate a second intermediate view synthesis result corresponding to a second one of the ordered plurality of views;

masking circuitry to mask the first intermediate view synthesis result with a first mask; and
merging circuitry to generate a final view synthesis result by merging the second intermediate view synthesis result with an output of the first mask.

16. The apparatus as defined in claim 15 , wherein the views reordering circuitry is to arrange the order of the plurality of views based on a distance between respective ones of the plurality of views.

17. The apparatus as defined in claim 15 , wherein the parameters of the target synthesized view include (a) position information and (b) orientation information.

18. The apparatus as defined in claim 15 , wherein the views reordering circuitry is to identify a degree of overlap between (a) the first one of the ordered plurality of views and the target synthesized view and (b) the second one of the ordered plurality of views and the target synthesized view.

19. The apparatus as defined in claim 18 , wherein the views reordering circuitry is to order the plurality of views based on one of the first one of the ordered plurality of views or the second one of the ordered plurality of views having a greater one of the degree of overlap.

20. The apparatus as defined in claim 15 , wherein the multi-pass view circuitry is to generate additional intermediate view synthesis results based on at least one of (a) a number of cameras corresponding to the plurality of views or (b) a number of positions of the cameras.

21. The apparatus as defined in claim 15 , wherein the plurality of views includes at least one of orthogonal position information or yaw-pitch-roll orientation information.

Assignees

• Intel Corp

Inventors

• Salahieh Basel
• Bhatia Sumit
• Boyce Jill

Classifications

• H04N13/282 Primary

  for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems · CPC title

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title

• G06T7/593

  from stereo images · CPC title

• H04N13/156

  Mixing image signals · CPC title

• H04N13/161

  Encoding, multiplexing or demultiplexing different image signal components (for multi-view video sequence encoding H04N19/597) · CPC title