Automated plant detection using image data

Patent metadata

Field	Value
Publication number	US-12530869-B2
Application number	US-202318352168-A
Country	US
Kind code	B2
Filing date	Jul 13, 2023
Priority date	May 9, 2017
Publication date	Jan 20, 2026
Grant date	Jan 20, 2026

Abstract

A plant treatment platform uses a plant detection model to detect plants as the plant treatment platform travels through a field. The plant treatment platform receives image data from a camera that captures images of plants (e.g., crops or weeds) growing in the field. The plant treatment platform applies pre-processing functions to the image data to prepare the image data for processing by the plant detection model. For example, the plant treatment platform may reformat the image data, adjust the resolution or aspect ratio, or crop the image data. The plant treatment platform applies the plant detection model to the pre-processed image data to generate bounding boxes for the plants. The plant treatment platform then can apply treatment to the plants based on the output of the machine-learned model.

First claim

What is claimed is:

1 . A farming machine comprising:
a treatment mechanism mounted to the farming machine, the treatment mechanism configured to treat areas in a field; an image sensor mounted to the farming machine, the image sensor configured toe capture images of the field; a computer; and a computer readable medium comprising instructions that, when executed by the computer, cause the computer to perform operations comprising:
treating areas in the field by the treatment mechanism mounted as the farming machine moves through or over the field;
receiving a first image of the field captured by the image sensor, the first image including image data representing areas of the field that received treatments by the treatment mechanism;
applying a treatment detection model to a second image based on the first image, the treatment detection model configured to:
generate a bounding box within the second image that encloses image data representing a first area of the areas of the field that received treatments by the treatment mechanism; and
determining, based on the bounding box enclosing the image data, the first area of the field received a treatment by the treatment mechanism.

2 . The farming machine of claim 1 , wherein the second image is a processed version of the first image.

3 . The farming machine of claim 1 , wherein the operations further comprise:
applying a plant detection model to the second image, the plant detection model configured to:
generate plant bounding boxes within the second image that enclose image data representing areas of the field with plants; and
treating, with the treatment mechanism, areas of the field that: (a) are enclosed by the plant bounding boxes in the second image and (b) exclude the first area.

4 . A method comprising:
treating areas in a field by a treatment mechanism mounted on a treatment platform moving through or over the field; receiving a first image of the field captured by an image sensor, the first image including image data representing areas of the field that received treatments by the treatment mechanism; applying a treatment detection model to a second image based on the first image, the treatment detection model configured to:
generate a bounding box within the second image that encloses image data representing a first area of the areas of the field that received treatments by the treatment mechanism; and
determining, based on the bounding box enclosing the image data, the first area of the field received a treatment by the treatment mechanism.

5 . The method of claim 4 , wherein the treatment detection model is further configured to generate a second bounding box within the second image that encloses image data representing a second area of the field that did not receive a treatment by the treatment mechanism.

6 . The method of claim 4 , wherein the treatment detection model is further configured to determine a confidence level that the bounding box is accurate.

7 . The method of claim 4 , wherein determining, based on the bounding box enclosing the image data, the first area of the field received the treatment by the treatment mechanism comprises:
determining a location of the bounding box in the second image; identifying a physical location of the first area in the field based on the location of the bounding box in the second image; and determining the first area of the field received the treatment based on the identified physical location.

8 . The method of claim 4 , wherein the second image is a processed version of the first image.

9 . The method of claim 4 , further comprising:
subsequent to determining the first area of the field received the treatment, treating additional areas of the field excluding the first area.

10 . The method of claim 4 , further comprising:
applying a plant detection model to the second image, the plant detection model configured to:
generate plant bounding boxes within the second image that enclose image data representing areas of the field with plants; and
treating, with the treatment mechanism, areas of the field that: (a) are enclosed by the plant bounding boxes in the second image and (b) exclude the first area.

11 . The method of claim 4 , wherein treating areas in the field comprises treating plants in the field by the treatment mechanism.

12 . The method of claim 4 , wherein treating areas in the field comprises spraying at least one of: plants or soil in the field.

13 . The method of claim 4 , wherein the bounding box within the second image encloses image data representing a patch of soil that received a treatment.

14 . A non-transitory computer-readable storage medium comprising stored instructions that, when executed by a computer, cause the computer to perform operations comprising:
treating areas in a field by a treatment mechanism mounted on a treatment platform moving through or over the field; receiving a first image of the field captured by an image sensor, the first image including image data representing areas of the field that received treatments by the treatment mechanism; applying a treatment detection model to a second image based on the first image, the treatment detection model configured to:
generate a bounding box within the second image that encloses image data representing a first area of the areas of the field that received treatments by the treatment mechanism; and
determining, based on the bounding box enclosing the image data, the first area of the field received a treatment by the treatment mechanism.

15 . The non-transitory computer-readable storage medium of claim 14 , wherein the treatment detection model is further configured to generate a second bounding box within the second image that encloses image data representing a second area of the field that did not receive a treatment by the treatment mechanism.

16 . The non-transitory computer-readable storage medium of claim 14 , wherein the treatment detection model is further configured to determine a confidence level that the bounding box is accurate.

17 . The non-transitory computer-readable storage medium of claim 14 , wherein determining, based on the bounding box enclosing the image data, the first area of the field received the treatment by the treatment mechanism comprises:
determining a location of the bounding box in the second image; identifying a physical location of the first area in the field based on the location of the bounding box in the second image; and determining the first area of the field received the treatment based on the identified physical location.

18 . The non-transitory computer-readable storage medium of claim 14 , wherein the second image is a processed version of the first image.

19 . The non-transitory computer-readable storage medium of claim 14 , further comprising:
subsequent to determining the first area of the field received the treatment, treating additional areas of the field excluding the first area.

20 . The non-transitory computer-readable storage medium of claim 14 , further comprising:
applying a plant detection model to the second image, the plant detection model configured to:
generate plant bounding boxes within the second image that enclose image data representing areas of the field with plants; and
treating, with the treatment mechanism, areas of the field that: (a) are enclosed by the plant bounding boxes in the second image and (b) exclude the first area.

Assignees

• Deere & Co

Inventors

• Redden Lee Kamp
• Padwick Christopher Grant
• Radhakrishnan Rajesh
• Ostrowski James Patrick

Classifications

• G06F18/2414

  Smoothing the distance, e.g. radial basis function networks [RBFN] · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06V10/451

  with interaction between the filter responses, e.g. cortical complex cells · CPC title

• G06V10/255

  Detecting or recognising potential candidate objects based on visual cues, e.g. shapes · CPC title

• G06V20/188

  Vegetation · CPC title