Variable focal length lens system with multi-level extended depth of field image processing

Patent metadata

Field	Value
Publication number	US-10101572-B2
Application number	US-201615360579-A
Country	US
Kind code	B2
Filing date	Nov 23, 2016
Priority date	Nov 23, 2016
Publication date	Oct 16, 2018
Grant date	Oct 16, 2018

Abstract

An imaging system including a tunable acoustic gradient (TAG) lens is associated with a user interface including a live display of an extended depth of field (EDOF) image. The TAG lens is utilized to produce a raw EDOF image, which may include defocus blur (e.g., resulting in part from the periodically modulated optical power variation of the TAG lens). The live EDOF image is repetitively updated in the user interface based on a translation state signal at a current time (e.g., indicating a speed of translation of the workpiece across a field of view of the imaging system, etc.) In response to a current state of the translation state signal, a corresponding type of live EDOF image of the workpiece is displayed in the user interface corresponding to an EDOF image data set that is based on a corresponding level of image processing to remove defocus blur.

First claim

What is claimed is:

1. A method for providing a live extended depth of field (EDOF) image of a workpiece displayed in a user interface associated with an imaging system, wherein the workpiece may translate across a field of view of the imaging system, the imaging system comprising:
a lens system including a tunable acoustic gradient (TAG) lens that is controlled to provide a periodically modulated optical power variation for the TAG lens and thereby provide a corresponding focus distance variation for the imaging system;
a control system comprising a lens controller configured to control the TAG lens;
a camera configured to input workpiece image light from the lens system at a plurality of focus distances for the imaging system and output a corresponding raw EDOF image data set corresponding to a raw EDOF image; and
an image processing portion configured to input a raw EDOF image data set and process it to provide an EDOF image data set that is based on image processing to remove defocus blur in a corresponding displayed EDOF image,
the method comprising:
repetitively operating the camera to output a sequence of the raw EDOF image data sets over time, corresponding to a sequence of raw EDOF images of the workpiece over time;
repetitively determining a translation state signal corresponding to a current translation speed of the workpiece across a field of view of the imaging system at a current time;
repetitively updating the live EDOF image of the workpiece displayed in the user interface based on a state of the translation state signal at a current time, wherein the repetitively updating the live EDOF image comprises:
in response to a first translation state signal corresponding to a first current translation speed, displaying a first type of live EDOF image of the workpiece in the user interface corresponding to an EDOF image data set that is based on a first level of image processing to remove defocus blur in the first type of live EDOF image; and
in response to a second translation state signal corresponding to a second current translation speed that is faster than the first current translation speed, displaying a second type of live EDOF image of the workpiece in the user interface corresponding to an EDOF image data set that is based on at least one of:
a second level of image processing to remove defocus blur in the second type of live EDOF image, wherein the second level of image processing corresponds to less image processing than the first level of image processing; or
no image processing to remove defocus blur in the second type of live EDOF image.

2. The method of claim 1 , wherein the second type of live EDOF image is a raw EDOF image corresponding to a raw EDOF image data set, and wherein the EDOF image data set is based on no image processing to remove defocus blur from the raw EDOF image used in the second type of live EDOF image.

3. The method of claim 1 , wherein:
the first level of image processing removes defocus blur at a first level of resolution in the first type of live EDOF image; and
the second level of image processing removes defocus blur at a second level of resolution in the second type of live EDOF image, wherein the second level of resolution is coarser than the first level of resolution.

4. The method of claim 3 , wherein the image processing that removes defocus blur comprises deconvolution processing of the raw EDOF image data set based on a predetermined integrated point spread function that characterizes operation of the lens system.

5. The method of claim 3 , wherein:
the first level of resolution is a full pixel-level resolution corresponding to a resolution of the raw EDOF image data set.

6. The method of claim 5 , wherein:
the second type of live EDOF image corresponds to the EDOF image data set that is based on a second level of image processing that removes defocus blur at the second level of resolution in the second type of live EDOF image; and
the second level of image processing that removes defocus blur at the second level of resolution is configured to:
process the raw EDOF image data to define a corresponding set of multi-pixel kernels;
at least one of average or combine pixel-level data of each multi-pixel kernel to provide a reduced amount of multi-pixel kernel level data that characterizes each multi-pixel kernel; and
process the multi-pixel kernel level data to remove defocus blur at a level of resolution corresponding to a size of the multi-pixel kernels in the second type of live EDOF image.

7. The method of claim 3 , wherein the repetitively updating the live EDOF image further comprises:
in response to a third translation state signal corresponding to a third current translation speed that is faster than the second current translation speed, displaying a third type of live EDOF image of the workpiece in the user interface corresponding to an EDOF image data set that is based on no image processing to remove defocus blur in the third type of live EDOF image.

8. The method of claim 1 , wherein the repetitive steps are performed at a rate such that the live EDOF image of a workpiece displayed in the user interface (UI) is repetitively updated at least 10 times per second, at least during operating states corresponding to the second translation state signal.

9. The method of claim 8 , wherein the repetitive steps are performed at a rate such that the live EDOF image of a workpiece displayed in the user interface (UI) is repetitively updated at least 25 times per second, at least during operating states corresponding to the second translation state signal.

10. The method of claim 1 , wherein:
the TAG lens is controlled to provide a periodically modulated optical power variation at a frequency of at least 3 KHz; and
the camera is configured to input workpiece image light from the lens system at a plurality of focus distances for the imaging system, during an exposure period spanning a plurality of periods of the periodically modulated optical power variation.

11. The method of claim 10 , wherein the plurality of focus distances are configured to span at least 70% of a maximum focus range of the imaging system.

12. The method of claim 11 , further comprising providing illumination having an intensity variation synchronized with the periodically modulated focus distance, such that a significant illumination intensity provides respective exposure contributions for focus distances within the at least 70% of the maximum focus range of the imaging system, and an insignificant illumination intensity suppresses exposure contributions for focus distances outside the at least 70% of the maximum focus range.

13. The method of claim 1 , wherein the determining of the translation state signal comprises at least one of:
determining a motion control speed;
determining a motion control speed in combination with a magnification level;
determining a speed of movement of a stage of the imaging system on which the workpiece is located; or
analyzing sequential images to determine an amount of translation of at least one feature in the images.

14. The method of claim 1 , wherein:
the displaying of the first type of live EDOF image is performed in response to a determination that at least one of the first translation state signal or the first current translation speed is below a speed-related threshold; and
the displaying of the second type of live EDOF image is performed in response to a determination that at least one of the second translation state signal or the second current translation speed is above the speed-related threshold.

15. The method of

Assignees

• Mitutoyo Corp

Inventors

• Bryll Robert Kamil
• Nagahama Tatsuya

Classifications

• G06T5/50 Primary

  using two or more images, e.g. averaging or subtraction · CPC title

• H04N23/67

  Focus control based on electronic image sensor signals · CPC title

• H04N23/951

  by using two or more images to influence resolution, frame rate or aspect ratio · CPC title

• H04N23/62

  Control of parameters via user interfaces · CPC title

• H04N23/676

  Bracketing for image capture at varying focusing conditions · CPC title