Localization method and helmet and computer readable storage medium using the same

Patent metadata

Field	Value
Publication number	US-11416719-B2
Application number	US-202017012016-A
Country	US
Kind code	B2
Filing date	Sep 3, 2020
Priority date	Dec 14, 2019
Publication date	Aug 16, 2022
Grant date	Aug 16, 2022

Abstract

The present disclosure provides a localization method as well as a helmet and a computer readable storage medium using the same. The method includes: extracting first feature points from a target image; obtaining inertial information of the carrier, and screening the first feature points based on the inertial information to obtain second feature points; triangulating the second feature points of the target image to generate corresponding initial three-dimensional map points, if the target image is a key frame image; performing a localization error loopback calibration on the initial three-dimensional map points according to at least a predetermined constraint condition to obtain target three-dimensional map points; and determining a positional point of the specific carrier according to the target three-dimensional map points. In this manner, the accuracy of the localization of a dynamic object such as a person when moving can be improved.

First claim

What is claimed is:

1. A computer-implemented localization method for a carrier, comprising steps of:
extracting first feature points from a target image;
obtaining inertial information of the carrier, and screening the first feature points based on the inertial information to obtain second feature points;
triangulating the second feature points of the target image to generate corresponding initial three-dimensional map points, in response to the target image being a key frame image;
performing a localization error loopback calibration on the initial three-dimensional map points according to at least a predetermined constraint condition to obtain target three-dimensional map points; and
determining a positional point of the carrier according to the target three-dimensional map points.

2. The method of claim 1 , wherein after the step of determining the positional point of the carrier according to the target three-dimensional map points further comprises steps of:
receiving a GPS positioning signal of the carrier;
extracting a localization result corresponding to the carrier in the GPS positioning signal;
fusing the localization result with the target three-dimensional map points to obtain first fusion positional points, in response to the strength of the GPS positioning signal being greater than a preset strength threshold; and
updating the positional point of the carrier according to the first fusion positional points.

3. The method of claim 1 , wherein the step of extracting the first feature points from the target image comprises:
extracting image feature points with a pixel gradient value greater than a preset pixel gradient threshold from the target image; and
selecting a predetermined number of the image feature points as the first feature points according to a distribution rule of the image feature points in a preset image plane.

4. The method of claim 1 , wherein the step of screening the first feature points based on the inertial information to obtain the second feature points comprises:
extracting reference feature points from a reference image;
calculating a photometric error value of the first feature points with respect to the reference feature point based on the inertial information; and
taking the first feature points as the second feature points, in response to the photometric error value being less than a preset photometric error threshold.

5. The method of claim 1 , further comprising steps of:
calculating a depth value of the second feature points of the target image, in response to the target image being a non-key frame image;
performing a two-dimensional mapping on the second feature points of the non-key frame image according to the depth value to generate corresponding two-dimensional map points;
fusing the target 3D map point and the 2D map points to obtain second fusion positional points; and
updating the positional point of the carrier according to the second fusion positional points.

6. The method of claim 1 , wherein the step of determining the positional point of the carrier according to the target three-dimensional map points comprises:
converting the target three-dimensional map points into a first coordinate in a preset localization coordinate system; and
converting the first coordinate into a corresponding positional point in a localization map.

7. The method of claim 1 , wherein after the step of determining the positional point of the carrier according to the target three-dimensional map points further comprises steps of:
transmitting the positional point of the carrier to a predetermined receiving terminal, so that the predetermined receiving terminal determines the position of the carrier according to the positional point of the carrier.

8. The method of claim 1 , wherein the target image is obtained through a camera.

9. The method of claim 1 , wherein the carrier is a helmet.

10. A helmet, comprising:
a memory;
a processor; and
one or more computer programs stored in the memory and executable on the processor, wherein the one or more computer programs comprise:
instructions for extracting first feature points from a target image;
instructions for obtaining inertial information of the helmet, and screening the first feature points based on the inertial information to obtain second feature points;
instructions for triangulating the second feature points of the target image to generate corresponding initial three-dimensional map points, in response to the target image being a key frame image;
instructions for performing a localization error loopback calibration on the initial three-dimensional map points according to at least a predetermined constraint condition to obtain target three-dimensional map points; and
instructions for determining a positional point of the helmet according to the target three-dimensional map points.

11. The helmet of claim 10 , wherein the one or more computer programs further comprise:
instructions for receiving a GPS positioning signal of the helmet;
instructions for extracting a localization result corresponding to the helmet in the GPS positioning signal;
instructions for fusing the localization result with the target three-dimensional map points to obtain first fusion positional points, in response to the strength of the GPS positioning signal being greater than a preset strength threshold; and
instructions for updating the positional point of the helmet according to the first fusion positional points.

12. The helmet of claim 10 , wherein the instructions for extracting the first feature points from the target image comprise:
instructions for extracting image feature points with a pixel gradient value greater than a preset pixel gradient threshold from the target image; and
instructions for selecting a predetermined number of the image feature points as the first feature points according to a distribution rule of the image feature points in a preset image plane.

13. The helmet of claim 10 , wherein the instructions for screening the first feature points based on the inertial information to obtain the second feature points comprise:
instructions for extracting reference feature points from a reference image;
instructions for calculating a photometric error value of the first feature points with respect to the reference feature point based on the inertial information; and
instructions for taking the first feature points as the second feature points, in response to the photometric error value being less than a preset photometric error threshold.

14. The helmet of claim 10 , wherein the one or more computer programs further comprise:
instructions for calculating a depth value of the second feature points of the target image, in response to the target image being a non-key frame image;
instructions for performing a two-dimensional mapping on the second feature points of the non-key frame image according to the depth value to generate corresponding two-dimensional map points;
instructions for fusing the target 3D map point and the 2D map points to obtain second fusion positional points; and
instructions for updating the positional point of the carrier helmet according to the second fusion positional points.

15. The helmet of claim 10 , wherein the instructions for determining the positional point of the helmet according to the target three-dimensional map points comprise:
instructions for converting the target three-dimensional map points into a first coordinate in a preset localization coordinate system; and
instructions for converting the first coordinate into a corresponding positional p

Assignees

• Ubtech Robotics Corp Ltd

Inventors

• Jiang Chenchen
• Liu Zhichao
• Zhao Yongsheng
• Tang Yu
• Pang Jianxin
• Xiong Youjun

Classifications

• G01S19/485 Primary

  whereby the further system is an optical system or imaging system · CPC title

• G06V10/44

  Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components · CPC title

• G06F18/25

  Fusion techniques · CPC title

• G01S19/49 Primary

  whereby the further system is an inertial position system, e.g. loosely-coupled · CPC title

• G01S19/47

  the supplementary measurement being an inertial measurement, e.g. tightly coupled inertial · CPC title