Pallet loading apparatus and pallet loading method

Patent metadata

Field	Value
Publication number	US-11964858-B2
Application number	US-202117357332-A
Country	US
Kind code	B2
Filing date	Jun 24, 2021
Priority date	Sep 23, 2020
Publication date	Apr 23, 2024
Grant date	Apr 23, 2024

Abstract

An embodiment pallet stacking apparatus includes a lidar installed in an unmanned forklift and configured to radiate laser light and convert range data reflected from a first cup-feet pallet to 3D point cloud data, a bird's eye view (BEV) conversion unit configured to convert the 3D point cloud data to a 2D BEV image, a cup recognition unit configured to perform channel normalization with input data of the 2D BEV image and recognize a cup position through calculation using convolutional neural network, a forklift controller configured to control operation of the unmanned forklift according to a control signal, and a control unit configured to identify a difference between a cup position of the first cup-feet pallet and a cup position of a stationary second cup-feet pallet and apply a control signal for adjustment to a matching position to the forklift controller.

First claim

What is claimed is:

1. A pallet stacking apparatus comprising:
a lidar installed in an unmanned forklift and configured to radiate laser light and convert range data reflected from a first cup-feet pallet to 3D point cloud data;
a forklift controller configured to control operation of the unmanned forklift;
at least one processor;
non-transitory computer readable medium storing program code that, when executed by the at least one processor, causes the at least one processor to:
convert the 3D point cloud data to a 2D BEV image;
perform channel normalization with input data of the 2D BEV image and recognize a cup position of the first cup-feet pallet through calculation using convolutional neural network (CNN);
measure a horizontal distance, a vertical distance, and a misaligned angle between a position of an upper cup of a second cup-feet pallet and a position of a lower cup of the first cup-feet pallet;
identify a difference between the position of the upper cup of the second cup-feet pallet and the position of the lower cup of the first cup-feet pallet; and
apply a control signal for adjustment to a matching position between the first cup-feet pallet and the second cup-feet pallet to the forklift controller on a fork of the unmanned forklift.

2. The pallet stacking apparatus of claim 1 , wherein the pallet stacking apparatus is configured to receive a transfer work instruction including at least one of a pallet ID, a transfer destination, a travel route, or to stack or not instruction.

3. The pallet stacking apparatus of claim 1 , wherein the lidar is installed at the fork so that a vertical position of the lidar will vary according to a raising or lowering operation of the fork.

4. The pallet stacking apparatus of claim 1 , wherein the forklift controller is configured to measure a distance between the cup position and the fork and control the unmanned forklift to lift up the first cup-feet pallet by the fork.

5. The pallet stacking apparatus of claim 1 , wherein the forklift controller is configured to match coordinates of the cup position of the first cup-feet pallet loaded on the fork with a coordinate system for a posture control of the fork and temporarily store the coordinates and vary the cup position according to a behavior of the unmanned forklift and raising or lowering control of the fork.

6. The pallet stacking apparatus of claim 1 , wherein the program code causes the at least one processor to generate the control signal to adjust the horizontal distance, the vertical distance, and the misaligned angle of the lower cup with respect to the upper cup.

7. The pallet stacking apparatus of claim 1 , wherein the program code causes the at least one processor to, when the horizontal distance, the vertical distance, and the misaligned angle do not satisfy a preset matching condition, apply the control signal to the forklift controller in a closed-loop control method.

8. The pallet stacking apparatus of claim 1 , wherein the program code causes the at least one processor to, when a matching condition of the horizontal distance and the vertical distance being within 20 mm and the misaligned angle being within 2° is satisfied, transmit a fork lowering signal to the forklift controller to load the cup-feet pallets in multi-stages.

9. A pallet stacking apparatus comprising:
a lidar installed in an unmanned forklift and configured to radiate laser light and convert range data reflected from a first cup-feet pallet to 3D point cloud data;
a forklift controller configured to control operation of the unmanned forklift;
at least one processor;
non-transitory computer readable medium storing program code that, when executed by the at least one processor, causes the at least one processor to:
convert the 3D point cloud data to a 2D BEV image by decreasing a height measurement value among a length, a width, and a height of the 3D point cloud data;
perform channel normalization with input data of the 2D BEV image and recognize a cup position of the first cup-feet pallet through calculation using a convolutional neural network (CNN);
identify a difference between the cup position of the first cup-feet pallet loaded on a fork of the unmanned forklift and a cup position of a stationary second cup-feet pallet; and
apply a control signal for adjustment to a matching position between the first cup-feet pallet and the second cup-feet pallet to the forklift controller.

10. The pallet stacking apparatus of claim 9 , wherein the program code causes the at least one processor to generate a 2D BEV image of an upper cup or a 2D BEV image of a lower cup based on the upper cup or the lower cup of the first cup-feet pallet, respectively, by decreasing the measurement value.

11. The pallet stacking apparatus of claim 9 , wherein the program code causes the at least one processor to differently convert the 2D BEV image generated for different measurement values through projection after translational movement, rotation, and size adjustment depending on an angle of the lidar.

12. A pallet stacking apparatus comprising:
a lidar installed in an unmanned forklift and configured to radiate laser light and convert range data reflected from a first cup-feet pallet to 3D point cloud data;
a forklift controller configured to control operation of the unmanned forklift;
at least one processor;
non-transitory computer readable medium storing program code that, when executed by the at least one processor, causes the at least one processor to:
convert the 3D point cloud data to a 2D BEV image;
perform channel normalization with input data of the 2D BEV image;
obtain an image region recognized as a cup in the 2D BEV image of a quadrangular boundary through image feature extraction and classification using a convolutional neural network (CNN); and
recognize a cup position of the first cup-feet pallet through the CNN by identifying a central point of the image region as the cup position;
identify a difference between the cup position of the first cup-feet pallet loaded on a fork of the unmanned forklift and a cup position of a stationary second cup-feet pallet; and
apply a control signal for adjustment to a matching position between the first cup-feet pallet and the second cup-feet pallet to the forklift controller.

13. The pallet stacking apparatus of claim 12 , wherein the lidar is installed at the fork so that a vertical position of the lidar will vary according to a raising or lowering operation of the fork.

14. The pallet stacking apparatus of claim 12 , wherein the forklift controller is configured to measure a distance between the cup position and the fork and control the unmanned forklift to lift up the first cup-feet pallet by the fork.

15. The pallet stacking apparatus of claim 12 , wherein the forklift controller is configured to match coordinates of the cup position of the first cup-feet pallet loaded on the fork with a coordinate system for a posture control of the fork and temporarily store the coordinates and vary the cup position according to a behavior of the unmanned forklift and raising or lowering control of the fork.

16. A method of operating an unmanned forklift, the method comprising:
transferring a first cup-feet pallet loaded on a fork of the unmanned forklift to a destination;
radiating laser light by using a lidar;
converting range data reflected from a stationary second cup-feet pallet to 3D point cloud data;
converting the 3D point cloud data to a 2D bird's-eye view (BEV) image;
performing channel normalization with input data of the 2D BEV image;
recognizing a cup positi

Assignees

• Hyundai Motor Co Ltd
• Kia Corp

Inventors

• Kim Joohan
• Jung Youngjin

Classifications

• G05D1/242

  Means based on the reflection of waves generated by the vehicle (using passive navigation aids external to the vehicle G05D1/244; using signals provided by artificial sources external to the vehicle G05D1/247) · CPC title

• B66F9/20

  Means for actuating or controlling masts, platforms, or forks (power take-off from vehicle transmissions, combined with vehicle propulsion systems B60K) · CPC title

• B66F9/063 Primary

  Automatically guided · CPC title

• B66F9/0755

  Position control; Position detectors · CPC title

• G05D1/024 Primary

  in combination with a laser (lasers per se H01S) · CPC title