Camera pose estimation techniques

Patent metadata

Field	Value
Publication number	US-11810322-B2
Application number	US-202117225396-A
Country	US
Kind code	B2
Filing date	Apr 8, 2021
Priority date	Apr 9, 2020
Publication date	Nov 7, 2023
Grant date	Nov 7, 2023

Abstract

Techniques are described for estimating pose of a camera located on a vehicle. An exemplary method of estimating camera pose includes obtaining, from a camera located on a vehicle, an image including a lane marker on a road on which the vehicle is driven, and estimating a pose of the camera such that the pose of the camera provides a best match according to a criterion between a first position of the lane marker determined from the image and a second position of the lane marker determined from a stored map of the road.

First claim

What is claimed is:

1. A method of estimating camera pose, comprising:
obtaining, from a camera located on a vehicle, an image comprising a lane marker on a road on which the vehicle is driven; and
estimating a pose of the camera such that the pose of the camera provides a best match according to a criterion between a first position of the lane marker determined from the image and a second position of the lane marker determined from a stored map of the road,
wherein the first position corresponds to pixel locations associated with a corner of the lane marker,
wherein the second position corresponds to a three-dimensional (3D) world coordinates of the corner of the lane marker,

wherein the second position of the lane marker is determined by:
obtaining, from the stored map and based on a location of the vehicle, a first set of one or more lane markers that are located within a pre-determined distance from the vehicle;
obtaining a second set of one or more lane markers from the first set of one or more lane markers based on a direction in which the vehicle is driven;
obtaining a third set of one or more lane markers from the second set of one or more lane markers based on a pre-determined field of view (FOV) of the camera; and
obtaining the second position of the lane marker from the third set of one or more lane markers;

wherein the best match according to the criterion is determined by minimizing a function of a combination of a cost of misalignment term and of a cost of constraint term,
wherein the cost of misalignment term is determined by minimizing a distance from the 3D world coordinates of the corner of the lane marker to the pixel locations associated with the corner of the lane marker, and
wherein the cost of constraint term is determined by minimizing a difference between a first estimated camera pose at a first time when the image is obtained and a second estimated camera pose from a second time, wherein the second time precedes in time the first time.

2. The method of claim 1 , wherein the distance is minimized by minimizing a sum of squared distance between the pixel locations associated with the corner of the lane marker and the 3D world coordinates of the corner of the lane marker.

3. The method of claim 1 , further comprising:
generating a binary image from the image obtained from the camera; and
generating a gray-scale image from the binary image, wherein the gray-scale image comprises pixels with corresponding values, wherein a value of each pixel is a second function of a second distance between a pixel location in the gray-scale image and the first position of the corner of lane marker in the gray-scale image.

4. The method of claim 1 , wherein the third set of one or more lane markers excludes one or more lane markers determined to be obstructed by one or more objects.

5. The method of claim 1 , wherein the cost of constraint term is another function of a first set of rotation values of the camera associated with the first time, a second set of translation values of the camera associated with the first time, a second difference between the first set of rotation values and a third set of rotation values of the camera associated with the second time, and a third difference between the second set of translation values and a fourth set of translation values of the camera associated with the second time.

6. A system comprising:
a processor; and
a memory that stores instructions executable by the processor to:
obtain, from a camera located on a vehicle, an image comprising a lane marker on a road on which the vehicle is driven; and
estimate a pose of the camera such that the pose of the camera provides a best match according to a criterion between a first position of the lane marker determined from the image and a second position of the lane marker determined from a stored map of the road,
wherein the first position corresponds to pixel locations associated with a corner of the lane marker,
wherein the second position corresponds to a three-dimensional (3D) world coordinates of the corner of the lane marker,

wherein the second position of the lane marker is determined by the processor configured to:
obtain, from the stored map and based on a location of the vehicle, a first set of one or more lane markers that are located within a pre-determined distance from the vehicle;
obtain a second set of one or more lane markers from the first set of one or more lane markers based on a direction in which the vehicle is driven;
obtain a third set of one or more lane markers from the second set of one or more lane markers based on a pre-determined field of view (FOV) of the camera; and
obtain the second position of the lane marker from the third set of one or more lane markers;

wherein the best match according to the criterion is determined by minimizing a function of a combination of a cost of misalignment term and of a cost of constraint term,
wherein the cost of misalignment term is determined by minimizing a distance from the 3D world coordinates of the corner of the lane marker to the pixel locations associated with the corner of the lane marker, and
wherein the cost of constraint term is determined by minimizing a difference between a first estimated camera pose at a first time when the image is obtained and a second estimated camera pose from a second time, wherein the second time precedes in time the first time.

7. The system of claim 6 , wherein the cost of constraint term represents a constraint that limits a search space.

8. The system of claim 6 , wherein the cost of constraint term is another function of a first set of rotation values of the camera associated with the first time, a second set of translation values of the camera associated with the first time, a second difference between the first set of rotation values and a third set of rotation values of the camera associated with the second time, and a third difference between the second set of translation values and a fourth set of translation values of the camera associated with the second time.

9. A non-transitory computer readable storage medium having code stored thereon, the code, when executed by a processor, causing the processor to implement a method, comprising:
obtaining, from a camera located on a vehicle, an image comprising a lane marker on a road on which the vehicle is driven; and
estimating a pose of the camera such that the pose of the camera provides a best match according to a criterion between a first position of the lane marker determined from the image and a second position of the lane marker determined from a stored map of the road,
wherein the first position corresponds to pixel locations associated with a corner of the lane marker,
wherein the second position corresponds to a three-dimensional (3D) world coordinates of the corner of the lane marker,

wherein the second position of the lane marker is determined by:
obtaining, from the stored map and based on a location of the vehicle, a first set of lane markers that are located within a pre-determined distance from the vehicle;
obtaining a second set of lane markers from the first set of lane markers based on a direction in which the vehicle is driven;
obtaining a third set of lane markers from the second set of lane markers based on a pre-determined field of view (FOV) of the camera; and
obtaining the second position of the lane marker from the third set of lane markers;

wherein the best match according to the criterion is determined by minimizing a function of a combination of a cost of misalignment term and of a cost of constraint term,
wherein the cost of misalignment term is determined by minimizing a distance from the 3D world coordinates

Assignees

• Tusimple Inc

Inventors

• Qian Chenzhe
• Luo Yi
• Xu Ke
• Wang Mingdong
• Hou Xiaodi

Classifications

• G06T7/74 Primary

  involving reference images or patches · CPC title

• G06T2207/30244

  Camera pose · CPC title

• G06T2207/30256

  Lane; Road marking · CPC title

• G06V20/588 Primary

  Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road · CPC title

• G06T2207/20081

  Training; Learning · CPC title