Electro-hydraulic varifocal lens-based method for tracking three-dimensional trajectory of moving object

Patent metadata

Field	Value
Publication number	US-12136239-B2
Application number	US-202217898861-A
Country	US
Kind code	B2
Filing date	Aug 30, 2022
Priority date	Aug 30, 2021
Publication date	Nov 5, 2024
Grant date	Nov 5, 2024

Abstract

The present disclosure discloses an electro-hydraulic varifocal lens-based method for tracking a 3D trajectory of a moving object. The method includes the following steps of: (1) obtaining a functional relation between a focusing control current and camera's intrinsic parameters; (2) obtaining a functional relation between focusing control currents of the electro-hydraulic varifocal lens and an optimal object distance; (3) initializing an object tracking algorithm, and taking an object tracking box as a subsequent focusing window; (4) carrying out first autofocusing, recording a focusing control current value after the autofocusing is completed, as well as a size and center point coordinates of the object tracking box; (5) calculating and recording coordinates of the object in 3D space; and (6) repeating steps (4) and (5) for the same object, and sequentially connecting the recorded coordinates of the object in 3D space into a trajectory.

First claim

What is claimed is:

1. An electro-hydraulic varifocal lens-based method for tracking a three-dimensional (3D) trajectory of a moving object, comprising:
step 1, calibrating an electro-hydraulic varifocal lens under different focal distances to obtain a functional relation between a focusing control current and a camera's intrinsic parameters;
step 2, establishing an electro-hydraulic varifocal lens-based optical imaging system model to obtain a functional relation between a focusing control current of the electro-hydraulic varifocal lens and an optimal object distance;
step 3, initializing an object tracking algorithm, generating an object tracking box, and selecting a to-be-tracked object;
step 4, carrying out autofocusing, and recording a focusing control current after the autofocusing is completed, as well as a size of the object tracking box in an image and center point coordinates after undistortion; wherein the autofocusing in step 4 comprises first autofocusing and subsequent autofocusing, and the first autofocusing specifically comprises:
searching an initial focusing control current at a certain stride, calculating a sharpness evaluation value of an internal image region of the object tracking box, obtaining a maximum sharpness evaluation value and a focusing control current corresponding to the maximum sharpness evaluation value, and setting a sharpness evaluation threshold:
K=αD max   (3)

wherein a denotes a preset sharpness confidence level, and α<1; K denotes a sharpness evaluation threshold used in the subsequent autofocusing; and D max denotes a maximum sharpness evaluation value; and
after the first autofocusing is finished, recording a size of the object tracking box in an image and center point coordinates after undistortion;

wherein the subsequent autofocusing specifically comprises:
calculating a sharpness evaluation value D i of the internal image region of the object tracking box; and
if D i >K, directly recording the focusing control current I i at this moment, as well as a size size i of the object tracking box in an image and center point coordinates after undistortion; or
if D i <K, reading a size size i of the object tracking box in the image at this moment, comparing the size with a size size i−1 of the object tracking box at last successful focusing, and adjusting a focusing control current to complete focusing; and after the focusing is completed, recording the focusing control current and the size of the object tracking box in the image after focusing and center point coordinates after undistortion;

step 5, calculating and recording, by a camera projection model, coordinates of the object in 3D space; and
wherein the camera projection model in step 5 is:
wherein (x i , y i ) denote center point coordinates of an object tracking box in an undistorted image, cx and cy denote coordinates of a camera's optical center on a pixel plane, s denotes a slant parameter between horizontal and vertical edges of a camera's photosensitive element, f xi , f yi , denote equivalent focal distances of a camera in x and y directions corresponding to a focusing control current I i at this moment respectively, and (X i , Y i , Z i ) denote 3D coordinates of a center point of a tracked object; and Z i =u i , wherein u i denotes an optimal object distance corresponding to the focusing control current Ii at this moment;
step 6, repeating steps 4-5 for the same tracked object, and sequentially connecting the recorded coordinates of the object in 3D space into a trajectory.

2. The electro-hydraulic varifocal lens-based method for tracking a 3D trajectory of a moving object according to claim 1 , wherein step 1 specifically comprises calibrating the electro-hydraulic varifocal lens under multiple focusing control currents to obtain the functional relation between the focusing control current and the camera's intrinsic parameters by curve fitting:
( f x ,f y )= H ( I )  (1)

wherein f x and f y denote equivalent focal distances of a camera in x and y directions of the pixel plane respectively, in a unit of px; and I denotes a focusing control current of an electro-hydraulic varifocal lens; and
obtaining coordinates of a camera's optical center on the pixel plane, and a slant parameter between horizontal and vertical edges of a camera's photosensitive element.

3. The electro-hydraulic varifocal lens-based method for tracking a 3D trajectory of a moving object according to claim 2 , wherein step 2 specifically comprises recording an optimal object distance under multiple focusing control currents by using the electro-hydraulic varifocal lens-based optical imaging system model, and conducting curve fitting on the recorded data to obtain a functional relation between focusing control currents of the electro-hydraulic varifocal lens and the optimal object distance:
u=F ( I )  (2)

wherein u denotes an optimal object distance, and I denotes a focusing control current of the electro-hydraulic varifocal lens.

4. The electro-hydraulic varifocal lens-based method for tracking a 3D trajectory of a moving object according to claim 1 , wherein when D i <K, the focusing control current is adjusted based on a comparison result of size i and size i−1 , which specifically comprises:
if size i <size i−1 , searching the focusing control current at a certain stride in the direction where the optimal object distance becomes longer, calculating a sharpness evaluation value in the object tracking box, and completing focusing after the sharpness evaluation value is greater than or equal to the sharpness evaluation threshold; or
if size i >size i−1 , searching the focusing control current at a certain stride in the direction where the optimal object distance becomes shorter, calculating a sharpness evaluation value in the object tracking box, and completing focusing after the sharpness evaluation value is greater than or equal to the sharpness evaluation threshold.

5. The electro-hydraulic varifocal lens-based method for tracking a 3D trajectory of a moving object according to claim 4 , wherein the undistortion in step 4 specifically comprises: calculating an undistorted image of a current frame according to a distortion model, and reading and recording center point coordinates of the object tracking box in the undistorted image.

6. The electro-hydraulic varifocal lens-based method for tracking a 3D trajectory of a moving object according to claim 5 , wherein the sharpness evaluation value is calculated using a Laplacian function, and the Laplacian function is expressed as:
D ( f )=Σ y Σ x |G ( x,y )|  (5)

wherein G(x, y) denotes convolution of a Laplacian operator at a pixel point (x, y), and the Laplacian operator is expressed as:

L
=

1
6

[

1

4

1

4

-
2

Assignees

• Univ Shanghai

Inventors

• Xie Shaorong
• Li Hengyu
• Liu Jingyi
• Wang Yueying
• Han Shuang
• Luo Jun

Classifications

• G02B7/28

  Systems for automatic generation of focusing signals · CPC title

• G02B3/14

  of variable focal length · CPC title

• H04N23/67

  Focus control based on electronic image sensor signals · CPC title

• H04N23/61

  based on recognised objects · CPC title

• G06T7/20

  Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) · CPC title