Industrial vehicle with feature-based localization and navigation

Patent metadata

Field	Value
Publication number	US-10473469-B2
Application number	US-201816179793-A
Country	US
Kind code	B2
Filing date	Nov 2, 2018
Priority date	Sep 4, 2015
Publication date	Nov 12, 2019
Grant date	Nov 12, 2019

Abstract

An industrial vehicle is provided comprising a drive mechanism, a steering mechanism, a vehicle controller, a camera, and a navigation module. The camera is communicatively coupled to the navigation module, the vehicle controller is responsive to commands from the navigation module, and the drive mechanism and the steering mechanism are responsive to commands from the vehicle controller. The camera is configured to capture an input image of a warehouse ceiling comprising elongated skylights characterized by different rates of image intensity change along longitudinal and transverse axial directions, and ceiling lights characterized by a circularly symmetric rate of image intensity change. The navigation module is configured to distinguish between the ceiling lights and the skylights and send commands to the vehicle controller for localization, or to navigate the industrial vehicle through the warehouse based upon valid ceiling light identification, valid skylight identification, or both.

First claim

What is claimed is:

1. A system comprising a vehicle camera, and a navigation module comprising memory and a processor, wherein:
the vehicle camera is communicatively coupled to the navigation module and is configured to capture input images;
the navigation module is configured to distinguish between object types in the input images by executing machine readable instructions to
run a cascading series of image smoothing and subsampling operations upon the input images to generate a collection of smoothed images,
build a first set of images from the collection of smoothed images, wherein the first set of images has a same size and structure as the smoothed images,
build a second set of images from the collection of smoothed images, wherein the second set of images has the same size and structure as the smoothed images,
utilize the first set of images to identify circular candidate object types in the input images,
utilize the second set of images to identify elongated candidate object types in the input images,
subject the identified circular and elongated candidate object types to candidate feature processing to identify valid object types in the input images, wherein the valid object types include circular object types which are ceiling lights and elongated object types which are skylights, and
generate navigational data based upon the identified valid object types.

2. The industrial vehicle of claim 1 wherein the collection of smoothed images is a Gaussian scale space pyramid.

3. The industrial vehicle of claim 2 wherein the Gaussian scale space pyramid is approximated by convolution with binomial filter kernels.

4. The industrial vehicle of claim 1 wherein the subsampling operations are implemented conditionally as a function of available navigation module computing power.

5. The industrial vehicle of claim 1 wherein
the first set of images is a determinant of Hessian response pyramid formed utilizing a determinant of Hessian; and
the second set of images is a trace of Hessian response pyramid formed utilizing a trace of Hessian.

6. The industrial vehicle of claim 5 wherein the determinant of Hessian response and trace of Hessian response are calculated based on each scale space image being convolved with second-order partial derivative filter kernels.

7. The industrial vehicle of claim 5 wherein the determinant of Hessian response is calculated by subtraction of a mixed second-order partial derivative term.

8. The industrial vehicle of claim 5 wherein the determinant of Hessian response suppresses responses to objects characterized by different rates of image intensity change along longitudinal and transverse axial directions.

9. The industrial vehicle of claim 8 wherein the determinant of Hessian response is utilized for multiscale non-maximum suppression wherein local maxima are located within a window comprising scale and spatial dimensions.

10. The industrial vehicle of claim 9 wherein an absolute minimum threshold is applied to prevent excessive noisy false positive determinant of Hessian responses.

11. The industrial vehicle of claim 5 wherein the determinant of Hessian response is utilized in a filtering keypoints function for removing candidate points that do not likely correspond to ceiling lights.

12. The industrial vehicle of claim 11 wherein the filtering keypoints function utilizes other filters on candidate points to check against empirically set thresholds.

13. The industrial vehicle of claim 12 wherein the other filters comprise scales at which keypoints are detected, a spatial location of a keypoint, a magnitude of the determinant of Hessian response at the keypoint location, a machine-learning classifier, or an average trace of Hessian response values of a surrounding area.

14. The industrial vehicle of claim 5 wherein the determinant of Hessian response is utilized in a refining keypoints function for refining spatial position and scale of ceiling light candidate keypoints.

15. The industrial vehicle of claim 5 wherein the trace of Hessian response is utilized in a sum large-scale trace of Hessian response images function for summing a selection of trace of Hessian response images from a trace of Hessian response pyramid into an integrated trace of Hessian response image.

16. The industrial vehicle of claim 15 wherein skylight regions are searched for within the trace of Hessian response pyramid.

17. The industrial vehicle of claim 5 wherein the trace of Hessian response is further processed by a threshold integrated trace of Hessian response function, wherein the trace of Hessian response is smoothed and a threshold is applied.

18. The industrial vehicle of claim 5 wherein the trace of Hessian response is utilized in a connected component filtering function for extracting and filtering connected components based on a binary thresholded integrated trace of Hessian response image and the connected component filtering function filters for size and aspect ratio to select substantially rectangular regions.

19. A navigation module comprising memory and a processor, wherein:
the memory is coupled to the processor; and
the processor is configured to distinguish between object types in input images by executing machine readable instructions to
run a cascading series of image smoothing and subsampling operations upon the input images to generate a collection of smoothed images,
build a first set of images from the collection of smoothed images, wherein the first set of images has a same size and structure as the smoothed images,
build a second set of images from the collection of smoothed images, wherein the second set of images has the same size and structure as the smoothed images,
utilize the first set of images to identify circular candidate object types in the input images,
utilize the second set of images to identify elongated candidate object types in the input images,
subject the identified circular and elongated candidate object types to candidate feature processing to identify valid object types in the input images, wherein the valid object types include circular object types which are ceiling lights and elongated object types which are skylights, and

generate navigational data based upon the identified valid object types.

Assignees

• Crown Equip Corp

Inventors

• Bell Mark

Classifications

• G06V10/462

  Salient features, e.g. scale invariant feature transforms [SIFT] · CPC title

• G06V10/52

  Scale-space analysis, e.g. wavelet analysis (multi-scale boundary representations G06V10/42) · CPC title

• G06T2207/30241

  Trajectory · CPC title

• G01C21/206 Primary

  specially adapted for indoor navigation · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title