Method and apparatus for accelerating hyperspectral video reconstruction

Patent metadata

Field	Value
Publication number	US-12014504-B2
Application number	US-202017626747-A
Country	US
Kind code	B2
Filing date	Jul 14, 2020
Priority date	Jul 15, 2019
Publication date	Jun 18, 2024
Grant date	Jun 18, 2024

Abstract

A method for accelerating hyperspectral video reconstruction includes steps of: acquiring, according to a spectral video and an RGB video captured by a hyperspectral video camera, a calibration matrix of the spectral video and the RGB video; sorting the calibration matrix to generate an ordered calibration matrix; converting, according to the ordered calibration matrix, the spectral video and the RGB video into a data matrix in a parallel manner; acquiring all related calibration points of a reconstruction region according to the ordered calibration matrix; and, reconstructing a hyperspectral video in a parallel manner according to the related calibration points and the data matrix. The related calibration points are acquired by sorting the calibration matrix, such that the number of times the calibration matrix is traverse is reduced, and the computation amount of hyperspectral video reconstruction is decreased.

First claim

The invention claimed is:

1. A method for accelerating hyperspectral video reconstruction, comprising steps of:
S 1 : acquiring, according to a spectral video and an RGB video captured by a hyperspectral video camera, a calibration matrix of the spectral video and the RGB video;
S 2 : sorting, according to a conditional constraint of spatial down-sampling in the hyperspectral video camera, the calibration matrix to generate an ordered calibration matrix;
S 3 : converting, according to the ordered calibration matrix, the spectral video and the RGB video into a data matrix in a parallel computing manner;
S 4 : acquiring all related calibration points of a reconstruction region according to the ordered calibration matrix; and
S 5 : reconstructing a hyperspectral video in a parallel computing manner according to the related calibration points and the data matrix.

2. The method for accelerating hyperspectral video reconstruction according to claim 1 , wherein step S 1 further comprises:
placing two-dimensional spatial coordinates of a first vertex of each calibration rectangle of the spectral video into a first-dimensional column vector, placing two-dimensional spatial coordinates of lithe a fourth vertex of each calibration rectangle of the spectral video into a second-dimensional column vector, and placing two-dimensional spatial coordinates of each calibration point of the RGB video into a third-dimensional column vector; and, combining the first-dimensional column vector, the second-dimensional column vector and the third-dimensional column vector to form a three-dimensional column vector matrix, and using the three-dimensional column vector matrix as a calibration matrix.

3. The method for accelerating hyperspectral video reconstruction according to claim 2 , wherein step S 2 further comprises:
distributing spatial down-sampling points of the hyperspectral video camera in the RGB video by using two-dimensional spatial coordinates (x,y), and sorting the calibration matrix according to a distribution rule of the spatial down-sampling points;
vertically sorting the entire calibration matrix by using a quick sorting algorithm by comparing a size of the y-coordinate value of the third-dimensional column vector;
transversely sorting the whole calibration matrix by the quick sorting algorithm by comparing a size of the x-coordinate value of the third-dimensional column vector;
generating two M×N ordered calibration matrices according to the number of rows M and the number of columns N of the spatial down-sampling points of the hyperspectral video camera;
placing a first ordered calibration matrix in calibration data of the spectral video as a spectral ordered calibration matrix;
putting the first-dimensional column vector and the second-dimensional column vector of the sorted calibration matrix in the spectral ordered calibration matrix;
setting a position where the spectral ordered calibration matrix does not contain the spatial down-sampling points of the hyperspectral video camera to be zero;
placing a second ordered calibration matrix in calibration data of the RGB video as an RGB ordered calibration matrix;
placing the third-dimensional column vector of the sorted calibration matrix in an RGB ordered calibration matrix; and
setting a position where the RGB ordered calibration matrix does not contain the spatial down-sampling points of the hyperspectral video camera to be zero;
according to the RGB ordered calibration matrix, computing transverse distance values between non-zero data points among half of mark points, and recording an average of the transverse distance values as a transverse distance between adjacent calibration points; and,
computing vertical distance values between non-zero data points among half of mark points, and recording an average of the vertical distance values as a vertical distance between adjacent calibration points.

4. The method for accelerating hyperspectral video reconstruction according to claim 3 , wherein step S 3 further comprises:
Acquiring a midpoint of a transverse point of each calibration rectangle according to the spectral ordered calibration matrix;
acquiring a vertical length of each calibration rectangle according to the spectral ordered calibration matrix; accelerating the generation of a spectral data matrix in a parallel computing manner in the spectral video; accelerating a synthesis of an RGB data matrix in a parallel computing manner according to the RGB video; and, combining the spectral data matrix and the RGB data matrix to form a data matrix.

5. The method for accelerating hyperspectral video reconstruction according to claim 4 , wherein step S 4 further comprises:
computing a reconstruction range of each reconstruction point according to the transverse distance between adjacent calibration points and the vertical distance between adjacent calibration points, and directly indexing the RGB ordered calibration matrix to obtain all related calibration points of each reconstruction point of the RGB data matrix.

6. An apparatus for accelerating hyperspectral video reconstruction, comprising:
A calibration matrix acquisition module configured to acquire a calibration matrix of a captured spectral video and RGB video;
a calibration matrix sorting module configured to sort the calibration matrix;
an adjacent calibration point vertical computation unit configured to compute an average transverse distance value between non-zero data points among half of mark points in an RGB ordered calibration matrix;
an adjacent calibration point transverse computation unit configured to compute an average vertical distance value between non-zero data points among half of mark points in the RGB ordered calibration matrix;
an ordered calibration matrix generation module configured to generate an ordered calibration matrix according to spatial down-sampling points of a hyperspectral video camera;
a data matrix generation module configured to convert the spectral video and the RGB video into a data matrix in a parallel manner according to the ordered calibration matrix;
a spectral data parallel computation unit configured to copy a spectral ordered calibration matrix and the spectral video into a parallel computation memory, and perform thread indexing to control the computation of each spectral data point;
an RGB data parallel computation unit configured to copy the RGB video into the parallel computation memory and perform thread indexing to control the computation of each RGB data point;
a calibration point acquisition module configured to acquire all related calibration points of a reconstruction region according to the ordered calibration matrix; and
a hyperspectral video reconstruction module configured to reconstruct a hyperspectral video in a parallel manner according to the related calibration points and the data matrix.

Assignees

• Nanjing University

Inventors

• Cao Xun
• Huang Erqi
• Chen Du
• Chen Linsen

Classifications

• H04N23/85

  for matrixing · CPC title

• H04N23/11

  for generating image signals from visible and infrared light wavelengths · CPC title

• G06T2207/10024

  Color image · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T3/40

  Scaling of whole images or parts thereof, e.g. expanding or contracting · CPC title