Thermal-depth fusion imaging

Patent metadata

Field	Value
Publication number	US-10796403-B2
Application number	US-201816131659-A
Country	US
Kind code	B2
Filing date	Sep 14, 2018
Priority date	Sep 14, 2017
Publication date	Oct 6, 2020
Grant date	Oct 6, 2020

Abstract

An imaging system is provided. The imaging system includes a 3D image capture device, which is configured to capture a depth image of an object, and a thermal image capture device, which is configured to capture a thermal image of the object. The imaging system also includes a processing system, which is coupled with the 3D image capture device and the thermal image capture device. The processing system is configured to process the depth image and the thermal image to produce a thermal-depth fusion image by aligning the thermal image with the depth image, and assigning a thermal value derived from the thermal image to a plurality of points of the depth image.

First claim

What is claimed is:

1. An imaging system, comprising:
a 3D image capture device, configured to capture a depth image of an object;
a thermal image capture device, configured to capture a thermal image of the object; and
a processing system, coupled with the 3D image capture device and the thermal image capture device, configured to process the depth image and the thermal image to produce a thermal-depth fusion image by:
aligning the thermal image with the depth image using a machine-learning process,
wherein localized deformation is iteratively applied to dense transformation estimates minimizing deformation error until convergence to produce a geometric transformation of the thermal image to match the depth image; and

assigning a thermal value derived from the thermal image to a plurality of points of the depth image.

2. The imaging system of claim 1 , wherein the machine-learning process includes per-pixel transformations established during an initial training process.

3. The imaging system of claim 1 , wherein the depth image is a 3D surface map of the object.

4. The imaging system of claim 1 , wherein the object is a human face and the processing system is further configured to:
compare the thermal-depth fusion image to a plurality of previously captured thermal-depth fusion images;
determine the closest match from the plurality of previously captured thermal-depth fusion images to the thermal-depth fusion image; and
based on the closest match, determine an identification of the human face.

5. The imaging system of claim 1 , wherein the 3D image capture device comprises two or more thermal image capture devices.

6. The imaging system of claim 1 , wherein the 3D image capture device is a time-of-flight sensor.

7. The imaging system of claim 3 , wherein the thermal-depth fusion image includes a curvature of the 3D surface map, a normal vector to a surface of the 3D surface map, and a temperature for a plurality of points on the 3D surface map.

8. The imaging system of claim 4 , wherein the human face is partially obscured, and the processing system uses a hierarchy of thermal signatures in determining the closest match.

9. The imaging system of claim 4 , wherein the processing system is further configured to:
associate a geometric shape, a surface normal, and a curvature to a thermal distribution with the human face; and
derive unique authentication signatures of the human face based on a combination of the plurality of previously captured thermal-depth fusion images to extract harmonic thermal-depth signatures comprising relations between curvature, thermal distributions, and facial structure of the human face.

10. The imaging system of claim 9 , wherein the processing system is further configured to:
monitor the changes in the geometric shape, the surface normal, and the principal curvature over time to contribute additional harmonic information to authentication signature of the human face.

11. A method of operating an imaging system, the method comprising:
receiving a depth image of an object from a 3D image capture device;
receiving a thermal image of the object from a thermal image capture device;
aligning the thermal image with the depth image using a machine-learning process,
wherein localized deformation is iteratively applied to dense transformation estimates minimizing deformation error until convergence to produce a geometric transformation of the thermal image to match the depth image; and

producing a thermal-depth fusion image by assigning a thermal value derived from the thermal image to a plurality of points of the depth image.

12. The method of claim 11 , wherein the machine-learning process includes per-pixel transformations established during an initial training process.

13. The method of claim 11 , wherein the depth image is a 3D surface map of the object.

14. The method of claim 11 , wherein the object is a human face and the method further comprising:
comparing the thermal-depth fusion image to a plurality of previously captured thermal-depth fusion images;
determining the closest match from the plurality of previously captured thermal-depth fusion images to the thermal-depth fusion image; and
based on the closest match, determining an identification of the human face.

15. The method of claim 11 , wherein the 3D image capture device comprises two or more thermal image capture devices.

16. The method of claim 13 , wherein the thermal-depth fusion image includes a curvature of the 3D surface map, a normal vector to a surface of the 3D surface map, and a temperature for a plurality of points on the 3D surface map.

17. The method of claim 14 , wherein the human face is partially obscured, and the processing system uses a hierarchy of thermal signatures in determining the closest match.

18. The method of claim 14 , further comprising:
associating a geometric shape, a surface normal, and a curvature to a thermal distribution with the human face; and
deriving unique authentication signatures of the human face based on a combination of the plurality of previously captured thermal-depth fusion images to extract harmonic thermal-depth signatures comprising relations between curvature, thermal distributions, and facial structure of the human face.

19. The method of claim 18 , further comprising:
monitoring the changes in the geometric shape, the surface normal, and the principal curvature over time to contribute additional harmonic information to authentication signature of the human face.

20. An imaging system, comprising:
a 3D image capture device, configured to capture a depth image of an object;
a thermal image capture device, configured to capture a thermal image of the object; and
a processing system, coupled with the 3D image capture device and the thermal image capture device, configured to process the depth image and the thermal image to produce a thermal-depth fusion image by:
aligning the thermal image with the depth image using a machine-learning process wherein localized deformation is iteratively applied to dense transformation estimates minimizing deformation error until convergence to produce a geometric transformation of the thermal image to match the depth image, wherein the machine-learning process includes per-pixel transformations established during an initial training process; and
assigning a thermal value derived from the thermal image to a plurality of points of the depth image.

Assignees

• Univ Colorado Regents

Inventors

• Choi Min-Hyung
• Transue Shane

Classifications

• G06V10/803

  of input or preprocessed data · CPC title

• G06F18/251

  of input or preprocessed data · CPC title

• G06V10/147

  Details of sensors, e.g. sensor lenses (fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19) · CPC title

• G06V40/172

  Classification, e.g. identification · CPC title

• G06V40/166

  using acquisition arrangements · CPC title