Detecting plugging of ground-engaging tools of an agricultural implement from imagery of a field using a machine-learned classification model

Patent metadata

Field	Value
Publication number	US-11250300-B2
Application number	US-201916723724-A
Country	US
Kind code	B2
Filing date	Dec 20, 2019
Priority date	Dec 20, 2019
Publication date	Feb 15, 2022
Grant date	Feb 15, 2022

Abstract

A computing system may be configured to perform operations including obtaining image data depicting a flow of soil around a ground-engaging tool of an agricultural implement as the ground-engaging tool is moved through the soil. Furthermore, the operations may include extracting a set of features from the obtained image data. Moreover, the operations may include inputting the set of features into the machine-learned classification model and receiving a soil flow classification of the set of features as an output of the machine-learned classification model. In addition, the operations may include determining when the ground-engaging tool is plugged based on the soil flow classification of the set of features.

First claim

The invention claimed is:

1. A computing system, comprising:
one or more processors; and
one or more non-transitory computer-readable media that collectively store:
a machine-learned classification model configured to receive input data and process the input data to output one or more soil flow classifications for the input data; and
instructions that, when executed by the one or more processors, configure the computing system to perform operations, the operations comprising:
obtaining image data depicting a flow of soil around a ground-engaging tool of an agricultural implement as the ground-engaging tool is moved through the soil;
identifying a plurality of soil units within the flow of soil around the ground-engaging tool depicted in the image data;
determining at least one of a size, a shape, a velocity, or a direction of travel of the plurality of soil units;
inputting the at least one of the size, the shape, the velocity, or the direction of travel of the plurality of soil units into the machine-learned classification model;
receiving a soil flow classification of the at least one of the size, the shape, the velocity, or the direction of travel of the plurality of soil units as an output of the machine-learned classification model; and
determining when the ground-engaging tool is plugged based on the soil flow classification of the at least one of the size, the shape, the velocity, or the direction of travel of the plurality of soil units.

2. The computing system of claim 1 , wherein:
the machine-learned classification model is configured to output a confidence score for the soil flow classification of the at least one of the size, the shape, the velocity, or the direction of travel of the plurality of soil units; and
determining when the ground-engaging tool is plugged comprises determining when the ground-engaging tool is plugged based on the confidence score and the soil flow classification of the at least one of the size, the shape, the velocity, or the direction of travel of the plurality of soil units.

3. The computing system of claim 1 , wherein the machine-learned classification model comprises a binary classifier configured to output a binary soil flow classification that classifies the at least one of the size, the shape, the velocity, or the direction of travel of the plurality of soil units determined from the image data as being associated with a plugged soil flow or a non-plugged soil flow.

4. The computing system of claim 1 , wherein the machine-learned classification model comprises a neural network.

5. The computing system of claim 1 , wherein the image data comprises a plurality of image frames.

6. The computing system of claim 5 , wherein determining the at least one of the size, the shape, the velocity, or the direction of travel of the plurality of soil units from the obtained image data comprises, for each image frame of the obtained image data, determining a corresponding at least one of the size, the shape, the velocity, or the direction of travel of the plurality of soil units.

7. The computing system of claim 1 , wherein the operations further comprise:
initiating display of an image frame of the image data from which the at least one of the size, the shape, the velocity, or the direction of travel of the plurality of soil units has been determined to an operator of the agricultural implement after the at least one of the size, the shape, the velocity, or the direction of travel of the plurality of soil units has been classified;
receiving an operator input associated with a correct soil flow classification for the at least one of the size, the shape, the velocity, or the direction of travel of the plurality of soil units; and
training the machine-learned classification model based on the received operator input.

8. The computing system of claim 1 , wherein, when it is determined that the ground-engaging tool is plugged, the operations further comprise initiating a control action associated with de-plugging the ground-engaging tool.

9. The computing system of claim 8 , wherein the control action comprises notifying an operator of the agricultural implement that the ground-engaging tool is plugged.

10. The computing system of claim 8 , wherein the control action comprises adjusting an operating parameter of the agricultural implement.

11. The computing system of claim 1 , wherein obtaining the image data comprises obtaining image data captured by an imaging device installed on the agricultural implement such that the ground-engaging tool is positioned within a field of view of the imaging device.

12. The computing system of claim 1 , wherein determining the at least one of the size, the shape, the velocity, or the direction of travel of the plurality of soil units comprises determining the size of the plurality of soil units.

13. The computing system of claim 1 , wherein determining the at least one of the size, the shape, the velocity, or the direction of travel of the plurality of soil units comprises determining the shape of the plurality of soil units.

14. The computing system of claim 1 , wherein determining the at least one of the size, the shape, the velocity, or the direction of travel of the plurality of soil units comprises determining the velocity of the plurality of soil units.

15. The computing system of claim 1 , wherein determining the at least one of the size, the shape, the velocity, or the direction of travel of the plurality of soil units comprises determining the direction of travel of the plurality of soil units.

16. A computer-implemented method, comprising:
obtaining, with a computing system comprising one or more computing devices, image data depicting a flow of soil around a ground-engaging tool of an agricultural implement as the ground-engaging tool is moved through the soil;
identifying, with the computing system, a plurality of soil units within the flow of soil around the ground-engaging tool depicted in the image data;
determining, with the computing system, at least one of a size, a shape, a velocity, or a direction of travel of the plurality of soil units;
inputting, with the computing system, the at least one of the size, the shape, the velocity, or the direction of travel of the plurality of soil units into a machine-learned classification model configured to configured to receive input data and process the input data to output one or more soil flow classifications for the at least one of the size, the shape, the velocity, or the direction of travel of the plurality of soil units;
receiving, with the computing system, a soil flow classification of the at least one of the size, the shape, the velocity, or the direction of travel of the plurality of soil units as an output of the machine-learned classification model; and
determining, with the computing system, when the ground-engaging tool is plugged based on the soil flow classification of the at least one of the size, the shape, the velocity, or the direction of travel of the plurality of soil units.

17. The computer-implemented method of claim 16 , wherein:
the machine-learned classification model is configured to output a confidence score for the soil flow classification of the at least one of the size, the shape, the velocity, or the direction of travel of the plurality of soil units; and
determining when the ground-engaging tool is plugged comprises determining, with the computing system, when the ground-engaging tool is plugged based on the confidence score and the soil flow classification of the at least one of the size, the shape, the

Assignees

• Cnh Ind Canada Ltd

Inventors

• STANHOPE Trevor
• Henry James W

Classifications

• A01B76/00 Primary

  Parts, details or accessories of agricultural machines or implements, not provided for in groups A01B51/00 - A01B75/00 · CPC title

• G06F18/241

  relating to the classification model, e.g. parametric or non-parametric approaches · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06F18/2178

  based on feedback of a supervisor · CPC title

• A01B23/06

  Discs; Scrapers for cleaning discs; Sharpening attachments · CPC title