Vehicle positioning or navigation utilizing associated feature pairs

Patent metadata

Field	Value
Publication number	US-9984467-B2
Application number	US-201514751375-A
Country	US
Kind code	B2
Filing date	Jun 26, 2015
Priority date	Jun 27, 2014
Publication date	May 29, 2018
Grant date	May 29, 2018

Abstract

According to the embodiments described herein, a method for vehicle positioning or navigation utilizing associated feature pairs may include creating a plurality of associated feature pairs by retrieving an initial set of camera data from the camera comprising two-dimensional UV space information, forming pairs from the UV space information, and associating each pair from the UV space information with pairs from each of the plurality of three-dimensional global feature points of the industrial facility map, calculating a best estimate poses from calculated vehicle poses of the associated feature pairs, using an accumulated odometry to update the best estimate pose to a current localized position and setting a seed position as the current localized position. The navigation of the materials handling vehicle is tracked and/or navigated along the inventory transit surface navigate in at least a partially automated manner utilizing the current localized position.

First claim

The invention claimed is:

1. A materials handling vehicle comprising a camera, one or more vehicular processors, a drive mechanism configured to move the materials handling vehicle along an inventory transit surface, a materials handling mechanism configured to store and retrieve goods in a storage bay of an industrial facility, and vehicle control architecture in communication with the drive and materials handling mechanisms, wherein:
the camera is communicatively coupled to the vehicular processor(s) and captures an input image of overhead features;
an industrial facility map comprises a plurality of three-dimensional overhead global feature points associated with a mapping of the overhead features; and
the vehicular processor(s) of the materials handling vehicle executes vehicle functions to:
(i) create a plurality of overhead associated feature pairs by:
retrieving an initial input image comprising overhead features from the camera comprising two-dimensional UV space information wherein U and V each correspond to a dimension in a two-dimensional space;
forming a plurality of overhead feature pairs by pairing points from the overhead features in the two-dimensional UV space information in the input image; and
associating the plurality of overhead feature pairs from the two-dimensional UV space information with overhead global feature point pairs formed from pairing the plurality of three-dimensional overhead global feature points of the industrial facility map, wherein a first overhead feature pair of the plurality of overhead feature pairs from the two-dimensional UV space information is associated with a first overhead global feature point pair formed from pairing a first pair of three-dimensional overhead global feature points such that the first overhead feature pair is associated with the first overhead global feature point pair to form a first overhead associated feature pair of the plurality of overhead associated feature pairs;

(ii) calculate a vehicle pose from the each of the plurality of overhead associated feature pairs within a visual range of the camera;
(iii) calculate a best estimate pose of the materials handling vehicle based on a highest match-rated pair of the plurality of overhead associated feature pairs by ranking matches between the plurality of overhead feature pairs from the two-dimensional UV space information and corresponding overhead global feature point pairs formed from the industrial facility map;
(iv) use an accumulated odometry of the materials handling vehicle to update the best estimate pose to a current localized position of the materials handling vehicle; and
(v) utilize the current localized position to: track the navigation of the materials handling vehicle along the inventory transit surface, or navigate the materials handling vehicle along the inventory transit surface in at least a partially automated manner, or both.

2. The materials handling vehicle of claim 1 , wherein the vehicular processor(s) further executes vehicle functions to validate the current localized position prior to updating the current localized position, such validation comprising functions to:
(i) record a number of unique global feature points after each data association step until the number is above a threshold such that the current localized position is validated.

3. The materials handling vehicle of claim 1 , wherein the vehicular processor(s) further executes vehicle functions to publish the current localized position on a display after updating the current localized position.

4. The materials handling vehicle of claim 1 , wherein the vehicular processor(s) further executes vehicle functions to:
(i) determine if the materials handling vehicle is lost;
(ii) if the materials handling vehicle is determined to be lost, retrieve a subsequent set of camera data from the camera and repeat functions (i)-(v) of claim 1 with the subsequent set of camera data from the camera in place of an initial set of camera data from the camera until the materials handling vehicle is found.

5. The materials handling vehicle of claim 4 , wherein determine if the materials handling vehicle is lost comprises determine if the materials handling vehicle is lost based on whether the current localized position comprises incorrect data.

6. The materials handling vehicle of claim 4 , the determination of whether the materials handling vehicle is lost is based at least in part on whether the camera data is insufficient to correct for an error associated with the accumulated odometry.

7. The materials handling vehicle of claim 4 , wherein the plurality of overhead associated feature pairs are created by data association steps comprising:
(i) creating UV space information pairs;
(ii) associating each global feature point with a first UV space feature in each UV space information pair; and
(iii) associating each global feature point within a maximum visual range of the camera with a second UV space feature in each UV space information pair.

8. The materials handling vehicle of claim 1 , wherein the calculation of a vehicle pose for each associated feature pair within a visual range of the camera is based at least in part on use of a global localization algorithm, wherein the global localization algorithm comprises a formula to calculate a yaw of the vehicle about a first UV space feature of an associated feature pair that comprises an anchored feature, wherein the formula comprises:
(i) N is a cross product of two 3D vectors;
(ii) T is a target 3D point for yaw rotation;

(
iii
)

⁢

⁢
D

=

N
x
2

*

N
y
2

(
iv
)

⁢

⁢

Y
offset

=

2
*

tan

-
1

(

N
x

N
x
2

+

N
y
2

+

N
y

)

Assignees

• Crown Equipment Ltd
• Crown Equip Corp

Inventors

• Chandrasekar Kashyap
• Estep Ryan
• Thomson Jacob
• Waltz Lucas

Classifications

• G01C21/206 Primary

  specially adapted for indoor navigation · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title

• G06T7/73

  using feature-based methods · CPC title

• G06T17/05

  Geographic models · CPC title