Work vehicle fork alignment system and method

Patent metadata

Field	Value
Publication number	US-12084330-B2
Application number	US-202117446222-A
Country	US
Kind code	B2
Filing date	Aug 27, 2021
Priority date	Aug 27, 2021
Publication date	Sep 10, 2024
Grant date	Sep 10, 2024

Abstract

A work vehicle comprising a pair of forks and an optical sensor. The optical sensor is configured to capture image data that includes the pair of forks and a moveable object. An electronic processor is configured to perform an operation by controllably adjusting the pair of forks, receive image data captured by the optical sensor, apply an artificial neural network to identify whether the pair of forks are aligned for moving the moveable object based on the image data, wherein the artificial neural network is trained to receive the image data as input and to produce as the output an indication of whether the pair of forks are aligned for moving the moveable object, access operation information corresponding to whether the pair of forks are aligned for moving the moveable object from a non-transitory computer-readable memory, and automatically adjust an operation of the work vehicle based on the operation information.

First claim

What is claimed is:

1. A method of operating a work vehicle, the method comprising:
capturing image data with an optical sensor coupled to the work vehicle wherein, the image data includes a pair of forks and a moveable object;
identifying whether the pair of forks are aligned for moving the moveable object and determining an alignment error between the pair of forks and the moveable object by processing the image data with an electronic processor;
accessing, from a non-transitory computer-readable memory, operation information corresponding to whether the pair of forks are aligned for moving the moveable object; and
adjusting an operation of the work vehicle based on the accessed operation information corresponding to whether the pair of forks are aligned for moving the moveable object and based on the alignment error between the pair of forks and the moveable object;
wherein the adjusting the operation of the work vehicle comprises transitioning the control of the work vehicle between a manual control and an automatic control.

2. The method of claim 1 , wherein the adjusting the operation of the work vehicle comprises adjusting a spacing of the pair of forks, side-shifting the pair of forks, raising the pair of forks upward, lowering the pair of forks downward, tipping upward the pair of forks, or tipping downward the pair of forks.

3. The method of claim 1 , wherein the optical sensor comprises a stereo camera.

4. The method of claim 1 , wherein the alignment error is displayed on an operator interface.

5. The method of claim 1 , wherein the optical sensor comprises a mono camera.

6. The method of claim 1 , wherein identifying whether the pair of forks are aligned for moving the moveable object by processing the image data comprises:
providing the image data as an input to an artificial neural network, wherein the artificial neural network is trained to receive as the input, image data including at least a portion of the pair of forks and at least a portion of the moveable object, and to produce as an output, an identification of whether the pair of forks are aligned for moving the moveable object; and
receiving an indication of the identification of whether the pair of forks are aligned for moving the moveable object as the output of the artificial neural network.

7. A control system for a work vehicle, the control system comprising:
an optical sensor coupled to the work vehicle, the optical sensor configured to capture image data that includes a pair of forks and a moveable object;
a non-transitory computer-readable memory storing operation information; and
an electronic processor configured to:
perform an operation by controllably adjusting a position of the pair of forks,
receive image data captured by the optical sensor,
apply an artificial neural network to identify whether the pair of forks are aligned for moving the moveable object based on the image data from the optical sensor, wherein the artificial neural network is trained to receive the image data as input and to produce as the output an indication of whether the pair of forks are aligned for moving the moveable object,
access, from the non-transitory computer-readable memory, the operation information corresponding to whether the pair of forks are aligned for moving the moveable object, and
adjust an operation of the work vehicle based on the accessed operation information corresponding to whether the pair of forks are aligned for moving the moveable object,
wherein the step of adjusting the operation of the work vehicle comprises a transition of the control of the work vehicle between a manual control and an automatic control.

8. The control system of claim 7 , wherein adjusting the operation of the work vehicle further comprises changing a spacing of the pair of forks, side-shifting the pair of forks, raising the pair of forks upward, lowering the pair of forks downward, tipping upward the pair of forks, or tipping downward the pair of forks.

9. The control system of claim 7 , wherein the optical sensor comprises a stereo camera.

10. The control system of claim 9 , wherein the stereo camera is configured to determine an alignment error between the pair of forks and the moveable object.

11. The control system of claim 10 , wherein the alignment error is displayed on an operator interface.

12. The control system of claim 10 , wherein adjusting the operation of the work vehicle is based on the operation information corresponding to whether the pair of forks are aligned for moving the moveable object and the alignment error between the pair of forks and the moveable object.

13. The control system of claim 7 , wherein the optical sensor comprises a mono camera.

14. The control system of claim 7 , wherein identifying whether the pair of forks are aligned for moving the moveable object by processing the image data comprises:
providing the image data as an input to an artificial neural network, wherein the artificial neural network is trained to receive as the input, image data including at least a portion of the pair of forks and at least a portion of the moveable object, and to produce as an output, an identification of whether the pair of forks are aligned for moving the moveable object; and
receiving an indication of the identification of whether the pair of forks are aligned for moving the moveable object as the output of the artificial neural network.

15. A work vehicle comprising:
a pair of forks;
an optical sensor coupled to the work vehicle, the optical sensor configured to capture image data that includes the pair of forks and a moveable object;
a non-transitory computer-readable memory storing operation information; and
an electronic processor configured to:
perform an operation by controllably adjusting a position of the pair of forks,
receive image data captured by the optical sensor,
apply an artificial neural network to identify whether the pair of forks are aligned for moving the moveable object, wherein the artificial neural network is trained to receive the image data as input and to produce as the output an indication of whether the pair of forks are aligned for moving the moveable object,
access, from the non-transitory computer-readable memory, the operation information corresponding to whether the pair of forks are aligned for moving the moveable object, and
adjust an operation of the work vehicle based on the accessed operation information corresponding to whether the pair of forks are aligned for moving the moveable object;

wherein the step of adjusting the operation of the work vehicle comprises a transition of the control of the work vehicle between a manual control and an automatic control.

16. The work vehicle of claim 15 , wherein the optical sensor comprises at least one of a mono camera or a stereo camera.

Assignees

• Deere & Co

Inventors

• Velde Todd F

Classifications

• G06F18/21

  Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation · CPC title

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• H04N13/204

  using stereoscopic image cameras (stereoscopic photography G03B35/00) · CPC title

• G05B13/027

  using neural networks only · CPC title