Adjusting stereo images

Patent metadata

Field	Value
Publication number	US-9369694-B2
Application number	US-201314018622-A
Country	US
Kind code	B2
Filing date	Sep 5, 2013
Priority date	Jun 11, 2010
Publication date	Jun 14, 2016
Grant date	Jun 14, 2016

Abstract

A computer-implemented method for adjusting stereo images includes receiving a video sequence associated with a recorded setting of a stereoscopic camera, the video sequence comprising pairs of left and right images. The method includes processing the pairs of left and right images to reduce influence of a discrepancy between the recorded setting and an actual configuration of the stereoscopic camera.

First claim

What is claimed is:

1. A computer-implemented method for processing stereo images, the method comprising:
receiving a video sequence associated with a recorded setting of a stereoscopic camera, the recorded setting including a desired setting of the stereoscopic camera, the video sequence comprising pairs of left and right images;
determining, by a computer, a corresponding actual setting of the stereoscopic camera, the corresponding actual setting including an actual setting of the stereoscopic camera while the video sequence is captured, wherein the corresponding actual setting is determined using a virtual three-dimensional (3D) space and a virtual camera rig in the virtual 3D space; and
processing the pairs of left and right images by the computer to detect and reduce an influence of a discrepancy between the recorded setting of the stereoscopic camera and the corresponding actual setting of the stereoscopic camera, wherein the discrepancy is detected by the computer using the virtual 3D space and the virtual camera rig in the virtual 3D space.

2. The computer-implemented method of claim 1 , further comprising:
including the left and right images in the virtual 3D space; and
generating the virtual camera rig in the virtual 3D space corresponding to the recorded setting of the stereoscopic camera; and
wherein the processing of the pairs of left and right images further comprises determining a virtual setting of the virtual camera rig based on the pairs of left and right images, and generating modified pairs of left and right images using the virtual camera rig.

3. The computer-implemented method of claim 1 , wherein the recorded setting of the stereoscopic camera includes values for the stereoscopic camera including at least one of: an interocular distance, a convergence distance, a focus distance, and left and right focal lengths.

4. The computer-implemented method of claim 3 , further comprising smoothing at least one of the values to reduce quantization artifacts originating from analog-to-digital conversion.

5. The computer-implemented method of claim 1 , further comprising adjusting the stereoscopic camera according to the processing.

6. The computer-implemented method of claim 2 , further comprising:
adjusting the virtual setting of the virtual camera rig; and
generating a modified video sequence using the virtual camera rig with the adjusted virtual setting.

7. The computer-implemented method of claim 6 , further comprising outputting the modified video sequence to a display device.

8. The computer-implemented method of claim 2 , further comprising:
generating at least one additional virtual camera rig in the virtual 3D space;
optimizing a first value of a virtual setting of a first virtual camera rig;
optimizing a second value of a virtual setting of a second virtual camera rig, wherein the second value is different from the first value.

9. The computer-implemented method of claim 8 , further comprising:
evaluating the first and second virtual camera rigs with regard to fit; and
selecting the virtual camera rig based on the evaluation.

10. A system comprising:
one or more processors;
memory communicatively linked with the one or more processors comprising code that when executed by the one or more processors causes the one or more processors to:
receive a video sequence associated with a recorded setting of a stereoscopic camera, the recorded setting including a desired setting of the stereoscopic camera, the video sequence comprising pairs of left and right images;
determine a corresponding actual setting of the stereoscopic camera, the corresponding actual setting including an actual setting of the stereoscopic camera while the video sequence is captured, wherein the corresponding actual setting is determined using a virtual three-dimensional (3D) space and a virtual camera rig in the virtual 3D space; and
process the pairs of left and right images to detect and reduce an influence of a discrepancy between the recorded setting of the stereoscopic camera and the corresponding actual setting of the stereoscopic camera, wherein the discrepancy is detected using the virtual 3D space and the virtual camera rig.

11. The system of claim 10 , wherein the code causes the one or more processors to:
include the left and right images in the virtual 3D space; and
generate the virtual camera rig in the virtual 3D space corresponding to the recorded setting of the stereoscopic camera; and
wherein the processing of the pairs of left and right images further comprises determining a virtual setting of the virtual camera rig based on the pairs of left and right images, and generating modified pairs of left and right images using the virtual camera rig.

12. The system of claim 10 , wherein the recorded setting of the stereoscopic camera includes values for the stereoscopic camera including at least one of: an interocular distance, a convergence distance, a focus distance, and left and right focal lengths.

13. The system of claim 12 , wherein at least one of the values is smoothed to reduce quantization artifacts originating from analog to digital conversion.

14. The system of claim 12 , wherein the code causes the one or more processors to transmit a signal to the stereoscopic camera to adjust a value of the recorded setting.

15. The system of claim 11 , wherein the code causes the one or more processors to:
adjust at least one value of the virtual setting of the virtual camera rig; and
generate a modified video sequence using the virtual camera rig with the adjusted virtual setting.

16. The system of claim 15 , wherein the modified video sequence is output to a display device.

17. The system of claim 12 , wherein the code causes the one or more processors to:
generate at least one additional virtual camera rig in the virtual 3D space;
optimize a first value of a virtual setting of a first virtual camera rig;
optimize a second value of a virtual setting of a second virtual camera rig, wherein the second value is different from the first value.

18. The system of claim 17 , further comprising code that when executed by the one or more processors causes the one or more processors to:
evaluate the first and second virtual camera rigs with regard to fit; and
select the virtual camera rig based on the evaluation.

19. A computer-program product tangibly embodied in a non-transitory computer-readable medium and comprising instructions that when executed by a processor perform a method comprising:
receiving a video sequence associated with a recorded setting of a stereoscopic camera, the recorded setting including a desired setting of the stereoscopic camera, the video sequence comprising pairs of left and right images;
determining a corresponding actual setting of the stereoscopic camera, the corresponding actual setting including an actual setting of the stereoscopic camera while the video sequence is captured, wherein the corresponding actual setting is determined using a virtual three-dimensional (3D) space and a virtual camera rig in the virtual 3D space; and
processing the pairs of left and right images to detect and reduce an influence of a discrepancy between the recorded setting of the stereoscopic camera and the corresponding actual setting of the stereoscopic camera, wherein the discrepancy is detected by the computer using the virtual 3D space and the virtual camera rig in the virtual 3D space.

20. The computer-program product of claim 19 , wherein the performed method further

Assignees

• Lucasfilm Entertainment Co Ltd

Inventors

• Levin John M
• Cantwell Brian
• Mallet Ronald
• Saltzman Jeffrey

Classifications

• H04N13/111

  Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation · CPC title

• G06T7/85

  Stereo camera calibration · CPC title

• G06T2207/30204

  Marker · CPC title

• H04N13/246 Primary

  Calibration of cameras · CPC title

• H04N13/0011

  Electricity · mapped topic