Autonomous vehicle, and object recognizing method in autonomous vehicle

Patent metadata

Field	Value
Publication number	US-9635346-B2
Application number	US-201514636406-A
Country	US
Kind code	B2
Filing date	Mar 3, 2015
Priority date	May 27, 2014
Publication date	Apr 25, 2017
Grant date	Apr 25, 2017

Abstract

An autonomous vehicle includes a travel vehicle main body, a model data storage, a photographic device, a search region determiner, an image feature point detector, a feature amount calculator and a position detector. The travel vehicle main body autonomously travels to a target position. The model data storage stores model data related to a geometric feature of an object. The photographic device photographs a periphery of the travel vehicle main body at the target position to acquire image data. The search region determiner predicts a position of the object based on the image data, and determines a search region of a predetermined range including the predicted position of the object. The image feature point detector detects a feature point of the image data with respect to the search region. The feature amount calculator calculates a feature amount of a matching candidate point extracted from the feature point. The position detector matches the feature amount of the matching candidate point with the model data to recognize the position of the object based on the image data.

First claim

What is claimed is:

1. An unmanned forklift comprising:
a pair of forks disposed on a front surface of the unmanned forklift for insertion into a pallet that includes inserting ports;
a travel vehicle main body that autonomously travels to a target position based on travel control information;
a model data storage that stores model data related to a geometric feature of the pallet;
a photographic device that photographs a periphery of the travel vehicle main body at the target position to acquire image data; and
a controller; wherein
the controller receives the travel control information including a travel path from a current position to the target position and a position information of the pallet;
the controller calculates current self-posture information of the unmanned forklift based on the travel control information;
the controller predicts a position of the pallet in a warehouse based on the image data, and determines a search region of a predetermined range that corresponds to a portion of the warehouse that includes the predicted position of the pallet within the warehouse based on the position information of the pallet and the current self-posture information;
the controller detects a feature point of the image data with respect to the search region;
the controller calculates a feature amount of a matching candidate point which is extracted from the feature point; and
the controller matches the feature amount of the matching candidate point with the model data to recognize the position of the pallet based on the image data.

2. The unmanned forklift according to claim 1 , wherein the controller extracts a plurality of feature points that satisfy a specific geometric condition from among feature points of the image data as matching candidate points.

3. The unmanned forklift according to claim 2 , wherein when one or more of the plurality of feature points that satisfy the specific geometric condition do not exist in the search region, the controller enlarges the search region to the periphery to detect the feature point.

4. The unmanned forklift according to claim 3 , wherein when a similarity degree with respect to the model data of the matching candidate point existing in the search region is less than a predetermined value, the controller enlarges the search region to the periphery and extracts the matching candidate point again.

5. The unmanned forklift according to claim 2 , wherein
the controller extracts, as the matching candidate points, three feature points appearing at an edge where three perpendicular planes of the pallet make contact with a floor surface.

6. The unmanned forklift according to claim 5 , wherein the specific geometric condition is defined by a distance between a first feature point and a second feature point, a distance between the first feature point and a third feature point, and a distance between the second feature point and the third feature point, the first feature point being positioned at a middle of the three feature points, and the second feature point and the third feature point being on both sides of the first feature point.

7. The unmanned forklift according to claim 6 , wherein the specific geometric condition includes an angle defined by a line segment connecting the first feature point and the second feature point and a line segment connecting the first feature point and the third feature point.

8. The unmanned forklift according to claim 2 , wherein the controller selects one or more interpolation points that satisfy a predetermined geometric condition with respect to the plurality of feature points, and extracts the plurality of feature points and the one or more interpolation points as the matching candidate points.

9. The unmanned forklift according to claim 1 , wherein the image data is three-dimensional image data.

10. The unmanned forklift according to claim 9 , wherein the photographic device is a stereo camera including two cameras that capture two-dimensional images, a feature point on the two-dimensional image is obtained based on the two two-dimensional images captured with the two cameras, and a distance image includes distance data captured with a distance sensor with respect to each feature point.

11. The unmanned forklift according to claim 9 , wherein the photographic device is a Time Of Flight camera that radiates an infrared light using an LED disposed at a periphery of the camera, measures a time until a reflected light reflected from the pallet is observed with the camera, and measures a distance to the pallet.

12. The unmanned forklift according to claim 2 , wherein the photographic device photographs in front of the pair of forks.

13. The unmanned forklift according to claim 12 , wherein the photographic device is disposed at a front end of one of the pair of forks.

14. The unmanned forklift according to claim 1 , wherein the controller extracts a point where color and shading change as a feature point for each pixel of the image data.

15. The unmanned forklift according to claim 1 , wherein the controller extracts the feature point based on a sum of squares of a luminance value using a Harris operator.

16. A pallet recognizing method for use with an unmanned forklift in a warehouse, the unmanned forklift including a pair of forks disposed on a front surface of the unmanned forklift and a travel vehicle main body, the pallet recognizing method comprising the steps of:
transmitting travel control information including a travel path from a current position to a target position and a position information of a pallet;
calculating current self-posture information of the unmanned forklift based on the travel control information;
causing the unmanned forklift to autonomously travel to the target position based on the travel control information;
storing model data related to a geometric feature of the pallet;
photographing a periphery of the travel vehicle main body at the target position to acquire image data;
predicting a position of the pallet based on the image data;
determining a search region of a predetermined range that corresponds to a portion of the warehouse that includes the predicted position of the pallet within the warehouse based on the position information of the pallet and the current self-posture information;
detecting a feature point of the image data with respect to the search region;
calculating a feature amount of a matching candidate points that is extracted from the feature point; and
matching the feature amount of the matching candidate points with the model data to recognize the position of the pallet based on the image data.

Assignees

• Murata Machinery Ltd

Inventors

• Iida Masaomi

Classifications

• H04N13/271

  wherein the generated image signals comprise depth maps or disparity maps · CPC title

• G06F2218/08

  Feature extraction · CPC title

• G01S17/86

  Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders · CPC title

• G01S17/894

  Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar · CPC title

• G01S7/4802

  using analysis of echo signal for target characterisation; Target signature; Target cross-section · CPC title