Positioning method based on lane line and feature point, electronic device, and storage medium

Patent metadata

Field	Value
Publication number	US-12430928-B2
Application number	US-202218088954-A
Country	US
Kind code	B2
Filing date	Dec 27, 2022
Priority date	Dec 29, 2021
Publication date	Sep 30, 2025
Grant date	Sep 30, 2025

Abstract

A positioning method based on a lane line and a feature point, an electronic device, and a storage medium, which relate to fields of computer, automatic driving, intelligent transportation, computer vision. The method may include: determining first real-time measurement residuals according to first sensor information of a movable object detected, the first real-time measurement residuals including a first inertial measurement unit measurement residual, a first lane line measurement residual and a first non-lane line measurement residual; updating a state vector of the movable object according to a kinematic model of an inertial measurement unit and the first real-time measurement residuals; and determining a pose of the movable object at a time instant corresponding to the updated state vector according to a pose vector in the updated state vector.

First claim

What is claimed is:

1. A positioning method based on a lane line and a feature point, the method comprising:
determining a plurality of first real-time measurement residuals according to a plurality of first sensor information of a movable object detected, wherein the first sensor information comprises a first inertial measurement unit information, a first lane line feature information and a first non-lane line feature point information, and the first real-time measurement residual comprises a first inertial measurement unit measurement residual, a first lane line measurement residual and a first non-lane line measurement residual;
updating a state vector of the movable object according to a kinematic model of an inertial measurement unit and the plurality of first real-time measurement residuals;
determining a pose of the movable object at a time instant corresponding to the updated state vector according to a pose vector in the updated state vector; and
controlling or positioning the movable object based on the determined pose of the movable object,
wherein the method further comprises:
determining a plurality of second real-time measurement residuals according to a plurality of second sensor information of the movable object detected, wherein the second sensor information comprises a second inertial measurement unit information, a second lane line feature information and a second non-lane line feature point information, and the second real-time measurement residual comprises a second inertial measurement unit measurement residual, a second lane line measurement residual and a second non-lane line measurement residual;
determining an initialization pose of the movable object according to the plurality of second real-time measurement residuals;
determining an initialization angular speedometer bias of the movable object according to the second lane line feature information and the second inertial measurement unit information;
determining an initialization wheel speedometer scale and/or an initialization accelerometer bias according to the initialization angular speedometer bias, the second inertial measurement unit information and the initialization pose; and
determining an initialized state vector according to the initialization pose, the initialization angular speedometer bias, the initialization wheel speedometer scale, and the initialization accelerometer bias.

2. The method according to claim 1 , wherein the determining an initialization wheel speedometer scale and/or an initialization accelerometer bias according to the initialization angular speedometer bias, the second inertial measurement unit information and the initialization pose comprises:
in response to the pose vector and the angular speedometer bias being initialized, determining an inertial measurement unit pre-integration model according to a first target velocity vector corresponding to a tenth sliding window, a second target velocity vector corresponding to an eleventh sliding window, a time difference formed by a time instant corresponding to the tenth sliding window and a time instant corresponding to the eleventh sliding window, a gravity vector, a wheel speedometer scale, a pose vector corresponding to the tenth sliding window, a pose vector corresponding to the eleventh sliding window, a rotation corresponding to the tenth sliding window, and a rotation corresponding to the eleventh sliding window,
determining a least square optimization model corresponding to the inertial measurement unit pre-integration model, wherein the least square optimization model comprises at least one selected from: a velocity vector to be estimated, a gravity vector to be estimated, or a wheel speedometer scale to be estimated, and
determining the initialization wheel speedometer scale and/or the initialization accelerometer bias according to a gravity vector of a preset length, the inertial measurement unit pre-integration model, and the least square optimization model.

3. The method according to claim 1 , wherein the first lane line measurement residual represents a distance residual formed by a line corresponding to a target first lane line feature point at a first sliding window and a feature point position corresponding to the target first lane line feature point at a second sliding window, and a time instant corresponding to the first sliding window is earlier than a time instant corresponding to the second sliding window; and
wherein the determining a plurality of first real-time measurement residuals according to a plurality of first sensor information of a movable object detected comprises:
determining a first pose corresponding to a first image frame and a second pose corresponding to a second image frame, wherein the first image frame corresponds to the first sliding window, and the second image frame corresponds to the second sliding window,
determining a first position information of the target first lane line feature point in a normalization plane corresponding to the second image frame, and
determining the first lane line measurement residual according to the first pose, the second pose, the first position information, and a predefined linear model.

4. The method according to claim 1 , wherein the first non-lane line measurement residual represents a re-projection residual of a target first non-lane line feature point in a third image frame corresponding to a third sliding window; and
wherein the determining a plurality of first real-time measurement residuals according to a plurality of first sensor information of a movable object detected comprises:
determining a first measurement value of a second position information of the target first non-lane line feature point in a normalization plane corresponding to the third image frame according to a world coordinate information of the target first non-lane line feature point in a world coordinate system,
determining a first rotation and a first translation between the world coordinate system and an inertial measurement unit coordinate system,
determining a first calculation value of the second position information of the target first non-lane line feature point in the normalization plane corresponding to the third image frame according to the world coordinate information, the first rotation, the first translation, an extrinsic parameter of a camera, and an extrinsic parameter of the inertial measurement unit, and
determining the first non-lane line measurement residual according to the first measurement value and the first calculation value.

5. The method according to claim 1 , wherein the first inertial measurement unit measurement residual comprises a pose measurement residual for the inertial measurement unit of the movable object in a stationary state; and
wherein the determining a plurality of first real-time measurement residuals according to a plurality of first sensor information of a movable object detected comprises:
determining a first heading angle corresponding to a fourth image frame collected by the movable object in the stationary state and a second heading angle corresponding to a fifth image frame collected by the movable object in the stationary state, wherein the fourth image frame corresponds to a fourth sliding window, the fifth image frame corresponds to a fifth sliding window, and the fourth sliding window is adjacent to the fifth sliding window,
determining, for the movable object in the stationary state, a second translation between an inertial measurement unit coordinate system at a time instant when the fourth sliding window is positioned and a world coordinate system, and a third translation between an inertial measurement unit coordinate system at a time instant when the fifth sliding window is positioned and the world coordinate system, and
determining

Assignees

• Beijing Baidu Netcom Sci & Tech Co Ltd

Inventors

• He Yuzhe
• Hou Shenhua
• Zhou Yao
• Peng Liang
• Wan Guowei

Classifications

• B60W2520/28

  Wheel speed · CPC title

• G06V20/58

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title

• B60W2554/4049

  Relationship among other objects, e.g. converging dynamic objects · CPC title

• B60W60/001

  Planning or execution of driving tasks · CPC title

• B60W2552/53

  Road markings, e.g. lane marker or crosswalk · CPC title