Active positioning method for bionic binocular vision during execution of SLAM task

Patent metadata

Field	Value
Publication number	US-12328510-B2
Application number	US-202318394344-A
Country	US
Kind code	B2
Filing date	Dec 22, 2023
Priority date	Aug 23, 2023
Publication date	Jun 10, 2025
Grant date	Jun 10, 2025

Abstract

Disclosed is an active positioning method for bionic binocular vision during execution of a simultaneous localization and mapping (SLAM) task. The method includes: capturing a panoramic image using a panoramic camera and detecting key scene information; assigning values to pixels in the panoramic image to obtain a panoramic value map; projecting field-of-view areas of left and right bionic eye cameras separately onto the panoramic value map, to obtain a current binocular field-of-view projection area; calculating a mean value of the current binocular field-of-view projection area in the panoramic value map; comparing the mean value of the current binocular field-of-view projection area with a value threshold, and obtaining a binocular field-of-view projection area with a mean value higher than the value threshold by moving the binocular bionic cameras; and finally using a high-value image captured by the left and right bionic eye cameras as an input of a SLAM system.

First claim

What is claimed is:

1. An active positioning method for bionic binocular vision during execution of a simultaneous localization and mapping (SLAM) task, comprising:
capturing a panoramic image using a panoramic camera and detecting key scene information in the panoramic image of a SLAM task;
assigning values to pixels in the panoramic image based on the key scene information, to obtain a panoramic value map;
projecting field-of-view areas of left and right bionic eye cameras separately onto the panoramic value map, to obtain a left field-of-view projection area and a right field-of-view projection area; controlling the left field-of-view projection area and the right field-of-view projection area to overlap with each other; and merging the left field-of-view projection area and the right field-of-view projection area to obtain a current binocular field-of-view projection area;
calculating a mean value of the current binocular field-of-view projection area in the panoramic value map;
comparing the mean value of the current binocular field-of-view projection area with a value threshold: if the mean value of the current binocular field-of-view projection area is greater than the value threshold, keeping the current binocular field-of-view projection area still, and using a high-value image currently captured by the left and right bionic eye cameras as an input for a SLAM system; and if the mean value of the current binocular field-of-view projection area is less than or equal to the value threshold, searching for a target binocular field-of-view projection area with a mean value higher than the value threshold in the panoramic value map; and
based on the current binocular field-of-view projection area and the target binocular field-of-view projection area, calculating required displacements for the left and right bionic eye cameras; moving the left and right bionic eye cameras according to the displacements, and finally using a high-value image captured by the left and right bionic eye cameras in the target binocular field-of-view projection area as an input for the SLAM system.

2. The active positioning method according to claim 1 , wherein the key scene information comprises feature points and dynamic objects; the feature points are extracted using a Features from Accelerated Segment Test (FAST) keypoint detection algorithm; and the dynamic objects are detected using a You Only Look Once (YOLOv7) detection algorithm, to obtain detection target boxes for the dynamic objects.

3. The active positioning method according to claim 2 , wherein assigning values to the pixels in the panoramic image based on the key scene information further comprises: assigning high pixel values to pixels located at feature points and pixels in the vicinity of the feature points in the panoramic image by using a pixel-wise assignment method, and assigning low pixel values to all pixels located within the detection target boxes for the dynamic objects, wherein the pixel values assigned to the pixels in the vicinity of feature points decrease as a neighborhood radius increases, and if a pixel is assigned multiple values, a maximum value assigned to the pixel is taken.

4. The active positioning method according to claim 3 , wherein projecting the field-of-view areas of the left and right bionic eye cameras separately onto the panoramic value map further comprises: calculating projection of the field-of-view areas of the left and right bionic eye cameras on the panoramic value map of the panoramic camera by using a projection equation of a normalized plane of the bionic eye camera and a projection equation of a normalized sphere of the panoramic camera.

5. The active positioning method according to claim 4 , wherein merging the left field-of-view projection area and the right field-of-view projection area further comprises: taking a union of the left field-of-view projection area and the right field-of-view projection area to obtain a union area; and generating a maximum bounding rectangle of the union area to obtain the current binocular field-of-view projection area.

6. The active positioning method according to claim 5 , wherein obtaining the mean value of the current binocular field-of-view projection area in the panoramic value map further comprises: based on value assignment results of the pixels in the panoramic value map, calculating a mean of values assigned to pixels within the current binocular field-of-view projection area to obtain the mean value of the current binocular field-of-view projection area.

7. The active positioning method according to claim 6 , wherein searching for the target binocular field-of-view projection area with the mean value higher than the value threshold in the panoramic value map further comprises:
generating a rectangular box that has the same size and at the same position as the current binocular field-of-view projection area in the panoramic value map;
translating the current rectangular box by a step size s from near to far, and calculating a mean value within the rectangular box after each translation; and
comparing the mean value within the translated rectangular box with the value threshold: if the mean value within the translated rectangular box is higher than the value threshold, marking the translated rectangular box as the target binocular field-of-view projection area; and if the mean value within the translated rectangular box is lower than or equal to the value threshold, and setting the current rectangular box as a last-translated rectangular box.

8. The active positioning method according to claim 7 , wherein, the displacement is an angular displacement, and a process of calculating the required displacements for the left and right bionic eye cameras further comprises: calculating a current pose of the left or right bionic eye camera in an initial pose coordinate system thereof by using the projection equation of the normalized plane of the left or right bionic eye camera; calculating, by using the projection equation from the normalized sphere of the panoramic camera to the pixel plane of the panoramic camera, a target pose of the left or right bionic eye camera for moving a field of view from the current binocular field-of-view projection area to the target binocular field-of-view projection area in the panoramic value map; and based on the current pose and the target pose of the left or right bionic eye camera, calculating a rotation matrix RcT between the two poses, to obtain the required displacement for moving the field of view of the left or right bionic eye camera from the current binocular field-of-view projection area to the target binocular field-of-view projection area in the panoramic value map.

9. An electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer program is executed by the processor to implement the active positioning method for bionic binocular vision during execution of a SLAM task according to claim 1 .

10. A non-transitory computer readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the active positioning method for bionic binocular vision during execution of a SLAM task according to claim 1 .

11. An electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer program is executed by the processor to implement the active positioning method for bionic binocular vision during execution of a SLAM task according to claim 2 .

12. An electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer program i

Assignees

• Univ Jining
• Univ Shanghai

Inventors

• Liu Jun
• Li Hengyu
• Wan Lei
• Liu Jingyi
• Wang Yueying
• Xie Shaorong
• Luo Jun

Classifications

• H04N2013/0081

  Depth or disparity estimation from stereoscopic image signals · CPC title

• H04N13/239

  using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence) · CPC title

• H04N13/296

  Synchronisation thereof; Control thereof · CPC title

• H04N13/271

  wherein the generated image signals comprise depth maps or disparity maps · CPC title

• H04N13/156

  Mixing image signals · CPC title