Industrial vehicles with overhead light based localization

Patent metadata

Field	Value
Publication number	US-9606540-B2
Application number	US-201514861626-A
Country	US
Kind code	B2
Filing date	Sep 22, 2015
Priority date	Sep 30, 2013
Publication date	Mar 28, 2017
Grant date	Mar 28, 2017

Abstract

According to the embodiments described herein, a method for environmental based localization may include capturing an input image of a ceiling comprising a plurality of skylights. Features can be extracted from the input image. The features can be grouped into a plurality of feature groups such that each of the feature groups is associated with one of the skylights. Line segments can be extracted from the features of each feature group, automatically, with one or more processors executing a feature extraction algorithm on each feature group separately. At least two selected lines of the line segments of each feature groups can be selected. A centerline for each of the feature groups can be determined based at least in part upon the two selected lines. The center line of each of the feature groups can be associated with one of the skylights.

First claim

What is claimed is:

1. An industrial vehicle comprising a camera, a steering apparatus, a throttle, wheels, and one or more processors, wherein
the steering apparatus controls the orientation of at least one of the wheels;
the throttle controls a traveling speed of the industrial vehicle;
the camera is communicatively coupled to the one or more processors;
the camera captures an input image of ceiling lights of the ceiling of the warehouse; and
the one or more processors execute machine readable instructions to
associate raw features of the ceiling lights of the input image with one or more feature groups,
execute a Hough transform to transform the raw features of the one or more feature groups into line segments associated with the one or more feature groups,
determine a convex hull of the raw features of the one or more feature groups,
compare the line segments of the one or more feature groups and the convex hull in Hough space,
discard the line segments of the one or more feature groups that are outside of a threshold of similarity to the convex hull of the raw features of the one or more feature groups, whereby a preferred set of lines is selected for the one or more feature groups from the line segments of the one or more feature groups,
determine a centerline of the one or more feature groups from the preferred set of lines,
associate the centerline of the one or more feature groups with one of the ceiling lights of the input image, and
navigate the industrial vehicle through the warehouse utilizing the steering apparatus, the throttle, and the centerline of the one or more feature groups.

2. The industrial vehicle of claim 1 , wherein the convex hull comprises hull line segments, and the line segments of the one or more feature groups are compared to the hull line segments.

3. The industrial vehicle of claim 2 , wherein the one or more processors execute the machine readable instructions to:
convert the hull line segments into Hough space coordinates, wherein the hull line segments are infinite lines represented by coordinates ρ and θ.

4. The industrial vehicle of claim 1 , wherein the one or more processors execute the machine readable instructions to:
rank the line segments of the one or more feature groups in order of strength; and
select a first edge line from the line segments of the preferred set of lines, wherein the first edge line is a highest ranked line of the line segments of the preferred set of lines.

5. The industrial vehicle of claim 4 , wherein the first edge line is represented by coordinates ρ and θ.

6. The industrial vehicle of claim 5 , wherein the one or more processors execute the machine readable instructions to:
select a second edge line from the line segments of the preferred set of lines, wherein the second edge line is selected based upon similarity to the θ of the first edge line.

7. The industrial vehicle of claim 6 , wherein the second edge line and the first edge line are separated by a distance threshold.

8. The industrial vehicle of claim 6 , wherein the one or more processors execute the machine readable instructions to:
search the line segments of the preferred set of lines from a high rank to a low rank to select the second edge line.

9. The industrial vehicle of claim 6 , wherein the one or more processors execute the machine readable instructions to:
find a vanishing point where the second edge line and the first edge line converge; and
calculate a line of bisection of the second edge line and the first edge line, wherein the centerline is calculated based upon the line of bisection.

10. The industrial vehicle of claim 1 , wherein each of the one or more feature groups of the raw features are transformed separately into the line segments.

11. The industrial vehicle of claim 1 , wherein the input image is underexposed to highlight the ceiling lights.

12. The industrial vehicle of claim 1 , wherein the ceiling lights comprise skylights, and the one or more processors execute the machine readable instructions to:
extract raw feature contours from the skylights, wherein the raw features comprise the raw feature contours;
classify the raw feature contours as belonging to a skylights class; and
group the raw feature contours into the one or more feature groups, wherein the one or more feature groups comprises one group per unique skylight of the skylights, and wherein each of the one group comprises the raw feature contours of the unique skylight.

13. The industrial vehicle of claim 12 , wherein the ceiling lights comprise round lights and merged lights, and the one or more processors execute the machine readable instructions to:
extract features from the round lights and the merged lights, wherein the raw features comprise the features from the round lights and the merged lights; and
classify the features from the round lights and the merged lights into a standard lights class and a merged lights class.

14. The industrial vehicle of claim 13 , wherein the feature extracted from each round light is a centroid of the round light.

15. The industrial vehicle of claim 13 , wherein the feature extracted from each merged light is two centroids of the merged light.

16. The industrial vehicle of claim 12 , wherein the raw features comprise unwanted features, and the one or more processors execute the machine readable instructions to:
classify the unwanted features as noise.

17. The industrial vehicle of claim 12 , wherein the raw feature contours are grouped into the one or more feature groups based upon relative proximity, and the one or more processors execute the machine readable instructions to:
calculate a minimum bounding rectangle for each of the raw feature contours; and
calculate the relative proximity based upon inter-feature distances of the minimum bounding rectangle for each of the raw feature contours.

18. The industrial vehicle of claim 1 , wherein the one or more processors execute machine readable instructions to remove lens distortion effects from the input image captured by the camera.

19. The industrial vehicle of claim 1 , wherein the one or more processors execute machine readable instructions to remove features outside of a region of interest from the input image.

20. The industrial vehicle of claim 1 , wherein the one or more processors execute machine readable instructions to convert frame coordinates of the input image to environment based localization frame coordinates.

21. An industrial vehicle comprising a camera, a steering apparatus, a throttle, wheels, and one or more processors wherein:
the steering apparatus controls the orientation of at least one of the wheels;
the throttle controls a traveling speed of the industrial vehicle;
the camera is communicatively coupled to the one or more processors;
the camera is mounted to the industrial vehicle and focused to a ceiling of a warehouse;
the camera captures an input image of a ceiling light of the ceiling of the warehouse; and
the one or more processors execute machine readable instructions to
extract raw feature contours from the ceiling light of the input image of the ceiling,
group the raw feature contours into a feature group,
execute a Hough transform to transform the raw feature contours of the feature group into line segments associated with the feature group,
determine a convex hull of the raw feature contours of the feature group,
compare the line segments of the feature group and the convex hull in Hough space,
dis

Assignees

• Crown Equipment Ltd
• Crown Equip Corp

Inventors

• Bell Mark
• Chandrasekar Kashyap
• Waltz Lucas B
• Thomson Jacob

Classifications

• G06V10/761

  Proximity, similarity or dissimilarity measures · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06F18/24

  Classification techniques · CPC title

• G06V10/42

  Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation · CPC title

• G06V10/48

  by mapping characteristic values of the pattern into a parameter space, e.g. Hough transformation · CPC title