System and method for advanced lens geometry fitting for imaging devices

Patent metadata

Field	Value
Publication number	US-9807359-B1
Application number	US-201715408923-A
Country	US
Kind code	B1
Filing date	Jan 18, 2017
Priority date	Nov 11, 2016
Publication date	Oct 31, 2017
Grant date	Oct 31, 2017

Abstract

A system and method for advanced lens geometry fitting is provided for projecting images using a projector with a non-planar lens. An initial estimate of intrinsic parameters of the projector can be determined, for example from a list; an initial estimate of extrinsic parameters of the projector can be determined based on the initial estimate of intrinsic parameters. The intrinsic parameters and extrinsic parameters can be optimized together by starting with a simple lens model and iteratively adding nonlinear terms, an error function evaluated at each iteration, for example for each set of intrinsic parameters in the list and the corresponding extrinsic parameters. As the values of the error function converges the optimized intrinsic parameters and the optimized extrinsic parameters can be used to generate images that are projected by the projector.

First claim

What is claimed is:

1. A system comprising:
a computing device; a projector comprising a non-planar lens, the projector configured for projection onto a object; the computing device configured to:
using an initial model of the non-planar lens that relates a position of projected points on the object to pixel points at an image modulator of the projector, determine at least one initial estimate of intrinsic parameters of the projector using the initial model;
determine at least one initial estimate of extrinsic parameters of the projector using the at least one initial estimate of the intrinsic parameters;
iteratively evaluate an error function that includes a comparison between the pixel points and a transformation of the projected points to the pixel points, an initial transformation determined using the at least one initial estimate of the intrinsic parameters and the at least one initial estimate of the extrinsic parameters, and iterative transformations including one or more of: varying one or more nonlinear terms used in the initial model of the non-planar lens, in one or more iterations; increasing a number of the one or more nonlinear terms used in the initial model of the non-planar lens, in one or more of the iterations;
varying intrinsic parameters of the projector; and varying extrinsic parameters of the projector;
when the error function, of one or more iterations, one or more of converges, reaches a minimum, and is less than a threshold value: store current value of the intrinsic parameters of the projector and the extrinsic parameters of the projector as pose data; and,
control the projector to project images onto the object based on the pose data.

2. The system of claim 1 , wherein the computing device is further configured to determine at least one initial estimate of intrinsic parameters of the projector using the initial model by determining a range of estimates for each of a plurality of the intrinsic parameters.

3. The system of claim 1 , wherein the computing device is further configured to determine at least one initial estimate of intrinsic parameters of the projector using the initial model by determining a set of discrete estimates of a plurality of the intrinsic parameters.

4. The system of claim 3 , wherein the computing device is further configured to determine the at least one initial estimate of extrinsic parameters for each of elements of the set of discrete estimates of the plurality of the intrinsic parameters.

5. The system of claim 4 , wherein the computing device is further configured to determine the at least one initial estimate of the extrinsic parameters, for each of the elements of the set of discrete estimates of the plurality of the intrinsic parameters, by extrapolating a line between a given projection point and a corresponding point at a curved image modulator that has a curvature determined from each of the elements of the set of discrete estimates of the plurality of the intrinsic parameters, to a pixel point on flat image modulator that shares a principal point with the curved image modulator.

6. The system of claim 5 , wherein the computing device is further configured to determine the at least one initial estimate of the extrinsic parameters using a direct linear transformation algorithm using a set of pixel points on the flat image modulator.

7. The system of claim 5 , wherein the computing device is further configured to iteratively evaluate the error function for each element of the set of discrete estimates of the plurality of the intrinsic parameters and corresponding estimates of the extrinsic parameters.

8. The system of claim 1 , wherein the non-planar lens comprises one or more of a fisheye lens and an f-theta lens, and wherein the initial model of the non-planar lens comprises one or more of a fisheye lens model and an f-theta lens model.

9. The system of claim 1 , wherein the initial model of the non-planar lens comprises one or more of: zero non-linear terms, one non-linear term, and a plurality of non-linear terms.

10. A method comprising:
at a computing device in communication with a projector comprising a non-planar lens, the projector configured for projection onto a object:
using an initial model of the non-planar lens that relates a position of projected points on the object to pixel points at an image modulator of the projector, determining, at the computing device, at least one initial estimate of intrinsic parameters of the projector using the initial model;
determining, at the computing device, at least one initial estimate of extrinsic parameters of the projector using the at least one initial estimate of the intrinsic parameters;
iteratively evaluating, at the computing device, an error function that includes a comparison between the pixel points and a transformation of the projected points to the pixel points, an initial transformation determined using the at least one initial estimate of the intrinsic parameters and the at least one initial estimate of the extrinsic parameters, and iterative transformations including one or more of: varying one or more nonlinear terms used in the initial model of the non-planar lens, in one or more iterations; increasing a number of the one or more nonlinear terms used in the initial model of the non-planar lens in one or more of the iterations; varying intrinsic parameters of the projector; and
varying extrinsic parameters of the projector;
when the error function, of one or more iterations, one or more of converges, reaches a minimum, and is less than a threshold value: store current value of the intrinsic parameters of the projector and the extrinsic parameters of the projector as pose data; and,
controlling, at the computing device, the projector to project images onto the object based on the pose data.

11. The method of claim 10 , further comprising: determining at least one initial estimate of intrinsic parameters of the projector using the initial model by determining a range of estimates for each of a plurality of the intrinsic parameters.

12. The method of claim 10 , further comprising: determining at least one initial estimate of intrinsic parameters of the projector using the initial model by determining a set of discrete estimates of a plurality of the intrinsic parameters.

13. The method of claim 12 , further comprising: determining the at least one initial estimate of extrinsic parameters for each element of the set of discrete estimates of the plurality of the intrinsic parameters.

14. The method of claim 13 , further comprising: determining the at least one initial estimate of the extrinsic parameters by extrapolating a line between a given projection point and a corresponding point at a curved image modulator that has a curvature similar to that of the non-planar lens, to a pixel point on flat image modulator that shares a principal point with the curved image modulator.

15. The method of claim 14 , further comprising: determining the at least one initial estimate of the extrinsic parameters using a direct linear transformation algorithm using a set of pixel points on the flat image modulator.

16. The method of claim 13 , further comprising: iteratively evaluating the error function for each element of the set of discrete estimates of the plurality of the intrinsic parameters and corresponding estimates of the extrinsic parameters.

17. The method of claim 10 , wherein the non-planar lens comprises one or more of a fisheye lens and an f-theta lens, and wherein the initial model of the non-planar lens comprises one or more of a fisheye lens model and

Assignees

• Christie Digital Systems Usa

Inventors

• Moule Kevin
• Ben-Lavi Eliezer Azi
• Scott Derek
• Urquhart Daniel Thomson

Classifications

• H04N9/3141 Primary

  Constructional details thereof (details not peculiar to the presence of an electronic spatial light modulator G03B21/14) · CPC title

• H04N9/3185 Primary

  Geometric adjustment, e.g. keystone or convergence (optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14) · CPC title

• G02B13/06

  Panoramic objectives; So-called "sky lenses" {including panoramic objectives having reflecting surfaces} · CPC title

• H04N9/3102

  using two-dimensional electronic spatial light modulators (micromechanical modulators as such G02B26/0833; liquid crystal modulators as such G02F1/13) · CPC title

• G02B27/0012

  Optical design, e.g. procedures, algorithms, optimisation routines · CPC title