Automated camera system control for harvesting machine unloading

Patent metadata

Field	Value
Publication number	US-12137629-B2
Application number	US-202318504642-A
Country	US
Kind code	B2
Filing date	Nov 8, 2023
Priority date	Sep 28, 2020
Publication date	Nov 12, 2024
Grant date	Nov 12, 2024

Abstract

A harvesting machine includes a header configured to gather harvested material into the harvesting machine during a harvesting operation, a conveyance subsystem configured to convey the harvested material from the harvesting machine to a receiving vehicle during the harvesting operation, an image capture system comprising at least one optical sensor, and a control system configured to determine a position of the receiving vehicle relative to the harvesting machine, determine an image magnification factor based on the determined position, and display, on a display device, an image of a portion of the receiving vehicle based on the image magnification factor.

First claim

What is claimed is:

1. A harvesting machine comprising:
a conveyance subsystem configured to perform an unloading operation that conveys harvested material from the harvesting machine to a receiving vehicle;
an image capture system comprising at least one optical sensor configured to capture an image representing the unloading operation; and
a control system configured to:
receive a receiving vehicle position signal indicative of a receiving vehicle position of the receiving vehicle;
receive a harvesting machine position signal indicative of a harvesting machine position of the harvesting machine;
determine a relative position of the receiving vehicle relative to the harvesting machine based on the receiving vehicle position signal and the harvesting machine position signal;
determine an image magnification factor based on the relative position of the receiving vehicle relative to the harvesting machine; and
generate a display of the image based on the image magnification factor.

2. The harvesting machine of claim 1 , wherein the at least one optical sensor generates a sensor signal, representing the image, comprising a set of pixels, and the control system is configured to perform a digital zoom comprising a selection of a subset of pixels, from the set of pixels, based on the image magnification factor.

3. The harvesting machine of claim 1 , wherein the at least one optical sensor comprises a camera having a variable focal length, and the control system is configured to control optical zoom of the camera to capture the image based on the image magnification factor.

4. The harvesting machine of claim 1 , wherein
the conveyance subsystem comprises a spout configured to convey the harvested material in an unloading direction, wherein the spout is movable between
a side-unloading configuration in which the unloading direction is oriented lateral to a direction of travel of the harvesting machine, and
a rear-unloading configuration in which the unloading direction is oriented to a rear of the harvesting machine, and

the control system is configured to configured to:
select a first image magnification factor for displaying the image based on a determination that the spout is in the side-unloading configuration, and
select a second image magnification factor, different than the first image magnification factor, for displaying the image based on a determination that the spout is in the rear-unloading configuration.

5. The harvesting machine of claim 4 , wherein
the receiving vehicle position signal is indicative of the receiving vehicle position in a coordinate system;
the harvesting machine position signal is indicative of the harvesting machine position of the harvesting machine in the coordinate system; and
the control system is configured to determine that the conveyance subsystem is in a particular one of the side-unloading configuration or the rear-unloading configuration based on the relative position of the receiving vehicle relative to the harvesting machine.

6. The harvesting machine of claim 4 , wherein the control system is configured to determine that the conveyance subsystem is in a particular one of the side-unloading configuration or the rear-unloading configuration based on at least one of:
a predicted crop landing location based on detected characteristics of the conveyance subsystem; or
a detected orientation of the spout.

7. The harvesting machine of claim 4 , wherein the control system is configured to determine that the conveyance subsystem is in a particular one of the side-unloading configuration or the rear-unloading configuration based on image processing performed on the image.

8. The harvesting machine of claim 1 , wherein the relative position of the receiving vehicle relative to the harvesting machine represents a distance between the harvesting machine and the receiving vehicle, and the image magnification factor is determined at least in in part based on the distance between the harvesting machine and the receiving vehicle.

9. The harvesting machine of claim 8 , wherein the control system is configured to:
detect a spout extension; and
determine the distance based on the spout extension.

10. The harvesting machine of claim 8 , wherein the control system is configured to update the image magnification factor based on changes to the distance between the harvesting machine and the receiving vehicle.

11. The harvesting machine of claim 1 , wherein the control system comprises:
an automatic cart filling control system configured to automatically control a position of the conveyance subsystem based on the image.

12. The harvesting machine of claim 1 , wherein the at least one optical sensor comprises a plurality of optical sensors with different field-of-view characteristics, and the control system is configured to:
select a particular optical sensor, from the plurality of optical sensors, based on the image magnification factor; and
receive the image from the particular optical sensor.

13. The harvesting machine of claim 1 , wherein the control system is configured to generate an image zoom user input mechanism based on the image magnification factor, display the image zoom user input mechanism on a display device, and control image zoom based on actuation of the image zoom user input mechanism.

14. A computer-implemented method of controlling a harvesting machine, the computer-implemented method comprising:
controlling a conveyance subsystem to perform an unloading operation that conveys harvested material from the harvesting machine to a receiving vehicle;
receiving one or more images representing the unloading operation;
determining a relative position of the receiving vehicle relative to the harvesting machine based on image processing performed on at least one image of the one or more images;
determining an image magnification factor based on the relative position; and
generating a display of at least one image of the one or more images based on the image magnification factor.

15. The computer-implemented method of claim 14 , wherein the generating the display comprises:
augmenting the display to identify at least one of a landing point or a height of the harvested material in the receiving vehicle.

16. The computer-implemented method of claim 14 , wherein
determining the relative position of the receiving vehicle comprises determining a distance between the harvesting machine and the receiving vehicle based on the image processing; and
determining the image magnification factor comprises determining the image magnification factor based on the distance between the harvesting machine and the receiving vehicle.

17. The computer-implemented method of claim 14 , further comprising:
selecting a particular optical sensor, from a plurality of optical sensors with different optical characteristics, based on the image magnification factor; and
receiving the image from the particular optical sensor.

18. The computer-implemented method of claim 14 , wherein controlling a display device to display the image comprises generating an image zoom user input mechanism based on the image magnification factor, displaying the image zoom user input mechanism on the display device, and controlling image zoom based on user actuation of the image zoom user input mechanism.

19. A harvester comprising:
an image processing system configured to process one or more images of at least a portion of a receiving vehicle;
an automatic vehicle filling control system configured to automaticall

Assignees

• Deere & Co

Inventors

• Faust Jeremy J
• O'Connor Kellen E

Classifications

• H04N23/69

  Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming · CPC title

• B65G2814/0302

  Central control devices · CPC title

• B65G67/22

  Loading moving vehicles · CPC title

• A01D90/02

  Loading means · CPC title

• A01D41/127 Primary

  Control or measuring arrangements specially adapted for combines · CPC title