Agricultural sprayer with real-time, on-machine target sensor

Patent metadata

Field	Value
Publication number	US-11944087-B2
Application number	US-202117172294-A
Country	US
Kind code	B2
Filing date	Feb 10, 2021
Priority date	Dec 21, 2020
Publication date	Apr 2, 2024
Grant date	Apr 2, 2024

Abstract

An agricultural applicator, that applies material to an agricultural field, includes an on-board, real-time image sensor that senses targets for the material to be applied. A controller controls applicators, such as nozzles or other applicators, to apply the material to the sensed targets.

First claim

What is claimed is:

1. An agricultural machine that travels across a field in a direction of travel, comprising:
a material reservoir;
a boom having an elongate axis that is transverse to an elongate axis of the agricultural machine;
a plurality of controllable valves mounted across the boom;
a plurality of optical sensors that each capture images of a portion of the field ahead of the plurality of controllable valves in the direction of travel;
a calibration system that, for each given optical sensor, generates a transform corresponding to the given optical sensor to apply to a received image, received by the given optical sensor to identify a location on the field corresponding to a position in the received image;
a pump that pumps material from the material reservoir to the plurality of controllable valves;
a target identification system that identifies a target in the received image and applies the transform to the received image to identify a location on the field corresponding to the target; and
a valve controller that generates a valve control signal to control the plurality of controllable valves to apply the material to the target.

2. The agricultural machine of claim 1 wherein the calibration system includes:
a boom position detector that detects a position of the boom relative to a set of optical calibration markers as a first boom calibration position; and
a calibration detection and processing system that obtains a first calibration image of the optical calibration markers captured by the given optical sensor and identifies the optical calibration markers in the first calibration image.

3. The agricultural machine of claim 2 wherein the calibration system includes:
a calibration transform generator that generates a first transform corresponding to the first boom calibration position based on the identified optical calibration markers in the first calibration image.

4. The agricultural machine of claim 3 and further comprising:
a calibration controller that controls movement of the boom to a plurality of different boom calibration positions, the boom position detector detecting a position of the boom at each of the boom calibration position, the calibration detection and processing system obtaining a plurality of different calibration images of the optical calibration markers, each calibration image corresponding to a different one of the plurality of different boom calibration positions, the calibration detection and processing system identifying the optical calibration markers in each of the plurality of different calibration images.

5. The agricultural machine of claim 4 wherein the calibration transform generator generates a transform corresponding to each of the plurality of different boom calibration positions based on the identified optical calibration markers in the corresponding calibration images.

6. The agricultural machine of claim 5 wherein the camera calibration system includes:
an interpolation system that interpolates among the transforms corresponding to the plurality of different boom calibration positions to obtain transforms corresponding to a plurality of different target boom positions, at least some of the target boom positions being different from the boom calibration positions.

7. The agricultural machine of claim 2 wherein the calibration detection and processing system includes:
a calibration mark identifier component that identifies a calibration mark in the first calibration image.

8. The agricultural machine of claim 7 wherein the calibration system includes:
a camera mount verification component that determines whether the given optical sensor is mounted in a right side up orientation based on a perspective indicated by the calibration mark.

9. The agricultural machine of claim 8 wherein the calibration mark identifier component identifies, as the calibration mark, a line identifier component that identifies a pair of elongate calibration marks and wherein the camera mount verification system is configured to determine whether an orientation of the given optical sensor on the boom is within a threshold of a desired orientation based on an orientation, in the first calibration image, of at least one of the elongate calibration markers in the pair of calibration markers.

10. The agricultural machine of claim 9 and further comprising:
a center marker identifier component that identifies a center marker centrally located between the pair of elongate calibration markers; and
an endpoint identifier component that identifies a first end point and a second endpoint of a first elongate calibration marker in the pair of calibration markers and a first endpoint and a second endpoint of a second elongate calibration marker in the pair of elongate calibration markers, wherein the camera mount verification component determines whether the given optical sensor is mounted in the right side up orientation based on a distance between the first endpoints of the first and second elongate calibration members and a distance between the second endpoints of the first and second elongate calibration members.

11. The agricultural machine of claim 10 wherein the endpoint identifier component identifies at least one of the first and second endpoints of the first and second elongate calibration markers based on a location of the center marker, relative to the first and second elongate calibration markers in the first calibration image.

12. The agricultural machine of claim 11 wherein each image processing module has a plurality of ports, one optical sensor being connected to each port, and wherein the calibration system includes:
a camera connection verification component that verifies that each optical sensor is connected to a correct port of the plurality of ports on the image processing module.

13. A method of controlling an agricultural machine that travels in a direction of travel and that has a movable boom mounted thereto, comprising:
detecting a position of the boom, relative to a set of optical calibration markers, as a first boom calibration position;
receiving a first calibration image of the optical calibration markers captured by an optical sensor mounted to the boom and positioned to capture images on a surface ahead of the boom in the direction of travel;
identifying the optical calibration markers in the first calibration image; and
generating a transform, corresponding to the optical sensor and the first boom calibration position, based on the calibration markers, the transform being operable to transform a position in the image of the surface to a geographic location on the surface.

14. The method of claim 13 and further comprising:
applying the transform to images received by the optical sensor to identify a location of material application targets; and
controlling a material application mechanism on the boom to apply material to the identified location of the material application targets.

15. The method of claim 14 and further comprising:
moving of the boom to a second boom calibration position;
detecting a position of the boom at the second boom calibration position;
receiving a second calibration image of the optical calibration markers corresponding to the second boom calibration position;
identifying the optical calibration markers in the second calibration image; and
generating a transform, corresponding to the optical sensor and the second boom calibration position, based on the identified optical calibration markers in the second calibration images.

16. The method of claim 15 and further comprising:
int

Assignees

• Deere & Co

Inventors

• Mizushima Akira
• Schlax Michael P
• Schoff Tucker J
• Rees Steven
• Humpal Richard A

Classifications

• A01M7/0089 Primary

  Regulating or controlling systems (the delivery being related to the movement of a vehicle B05B9/06) · CPC title

• G06V20/188

  Vegetation · CPC title

• A01M7/0057

  with active regulation of the boom position · CPC title