Mapping field anomalies using digital images and machine learning models

Patent metadata

Field	Value
Publication number	US-12008744-B2
Application number	US-201916707355-A
Country	US
Kind code	B2
Filing date	Dec 9, 2019
Priority date	Dec 10, 2018
Publication date	Jun 11, 2024
Grant date	Jun 11, 2024

Abstract

A computer-implemented method for generating an improved map of field anomalies using digital images and machine learning models is disclosed. In an embodiment, a method comprises: obtaining a shapefile that defines boundaries of an agricultural plot and boundaries of the field containing the plot; obtaining a plurality of plot images within the field from one or more image capturing devices that are located within the boundaries of the field; calibrating and pre-processing the plurality of plot images to create a plot map of the agricultural plot at a plot level; based on the plot map of the agricultural plot, generating a plot grid; based on the plot grid and the plot map, generating a plurality of plot tiles; based on the plurality of plot tiles, generating, using a first machine learning model and a plurality of first image classifiers corresponding to one or more first anomalies, a set of classified plot images that depicts at least one anomaly; based on the set of classified plot images, generating a plot anomaly map for the agricultural plot; transmitting the plot anomaly map to one or more controllers that control one or more agricultural machines or database systems to perform agricultural functions on the agricultural plot.

First claim

What is claimed is:

1. A computer-implemented method for generating an improved map of field anomalies using digital images and machine learning models, the method comprising:
obtaining a shapefile that defines boundaries of an agricultural plot;
based on the shapefile, obtaining a plurality of ground based plot images from one or more image capturing devices mounted at a fixed ground location or a ground vehicle at the agricultural plot;
calibrating the plurality of ground based plot images;
stitching the plurality of calibrated ground based plot images into a plot map of the agricultural plot at a plot level;
generating a plot grid;
based on the plot grid and the plot map, defining a plurality of plot tiles for the agricultural plot, each of the plurality of plot tiles including multiple pixels of the plurality of calibrated ground based plot images;
classifying the plurality of plot tiles, using a first machine learning model and a plurality of first image classifiers corresponding to one or more first anomalies, into a set of classified plot images that depicts at least one anomaly, wherein each of the plurality of plot tiles is classified into classifications at least corresponding to a crop, a weed, trees, and inter-row damage;
determining, for each image in the set of classified plot images, a probability that the image is correctly classified, and further comparing the probability to an acceptable probability;
based on the set of classified plot images, generating a plot anomaly map for the agricultural plot; and
transmitting the plot anomaly map to one or more controllers that control one or more agricultural machines to perform agricultural functions on the agricultural plot.

2. The computer-implemented method of claim 1 , wherein the shapefile is generated by performing:
obtaining a plurality of aerial images of an agricultural field;
calibrating the plurality of aerial images stitching the plurality of calibrated aerial images into a field map of the agricultural field at a field level;
based on the field map of the agricultural field, generating a field grid;
based on the field grid and the field map, defining a plurality of field tiles;
classifying the plurality of field tiles, using a second machine learning model and a plurality of second image classifiers corresponding to one or more second anomalies, into a set of classified field images that depicts at least one field anomaly;
based on the set of classified field images, generating a field anomaly map for the agricultural field; and
based on the field anomaly map, generating the boundaries for the agricultural plot defined in the shapefile.

3. The computer-implemented method of claim 2 , wherein the agricultural plot is a part of the agricultural field.

4. The computer-implemented method of claim 2 , wherein the plot anomaly map has a higher-level of detail than the field anomaly map;
wherein the plurality of first image classifiers has a higher-level of detail than the plurality of second image classifiers; and
wherein the plurality of first image classifiers includes two or more of: one or more interrow image classifiers, one or more weed image classifiers, one or more bare soil classifiers, one or more lodging classifiers, or one or more standing water classifiers; and
wherein the one or more first anomalies have a higher-level of detail than the one or more second anomalies.

5. The computer-implemented method of claim 1 , wherein the shapefile is used to control the one or more image capturing devices configured to capture the plurality of ground based plot images from the agricultural plot defined by the boundaries.

6. The computer-implemented method of claim 1 , wherein the plurality of ground based plot images is captured by the one or more image capturing devices as the one or more image capturing devices are controlled based on contents of the shapefile specifying the boundaries of the agricultural plot.

7. The computer-implemented method of claim 1 , wherein the one or more image capturing devices are installed on any of: moving farming equipment, stationary farming equipment, stationary posts, stationary structures, handheld devices, or mobile devices.

8. The computer-implemented method of claim 1 , wherein the calibrating of the plurality of ground based plot images comprises correcting one or more colors depicted in the plurality of ground based plot images.

9. The computer-implemented method of claim 1 , wherein the plot anomaly map for the agricultural plot comprises one or more specific anomaly maps, each specific anomaly map depicting a specific anomaly identified for the agricultural plot.

10. One or more non-transitory storage media storing instructions which, when executed by one or more computing devices, cause the one or more computing devices to perform:
obtaining a shapefile that defines boundaries of an agricultural plot;
based on the shapefile, obtaining a plurality of ground based plot images from one or more image capturing devices mounted at a fixed ground location or a ground vehicle at the agricultural plot;
calibrating the plurality of ground based plot images;
stitching the plurality of calibrated ground based plot images into a plot map of the agricultural plot at a plot level;
generating a plot grid;
based on the plot grid and the plot map, defining a plurality of plot tiles for the agricultural plot, each of the plurality of plot tiles including multiple pixels of the plurality of calibrated ground based plot images;
classifying the plurality of plot tiles using a first machine learning model and a plurality of first image classifiers corresponding to one or more first anomalies, into a set of classified plot images that depicts at least one anomaly, wherein each of the plurality of plot tiles is classified into classifications at least corresponding to a crop, a weed, trees, and inter-row damage;
determining, for each image in the set of classified plot images, a probability that the image is correctly classified, and further comparing the probability to an acceptable probability;
based on the set of classified plot images, generating a plot anomaly map for the agricultural plot; and
transmitting the plot anomaly map to one or more controllers that control one or more agricultural machines to perform agricultural functions on the agricultural plot.

11. The one or more non-transitory storage media of claim 10 , storing additional instructions for generating the shapefile, which, when executed by the one or more computing devices, cause the one or more computing devices to perform:
obtaining a plurality of aerial images of an agricultural field;
calibrating the plurality of aerial images
stitching the plurality of calibrated aerial images into a field map of the agricultural field at a field level;
based on the field map of the agricultural field, generating a field grid;
based on the field grid and the field map, defining a plurality of field tiles;
classifying the plurality of field tiles using a second machine learning model and a plurality of second image classifiers corresponding to one or more second anomalies, into a set of classified field images that depicts at least one field anomaly;
based on the set of classified field images, generating a field anomaly map for the agricultural field; and
based on the field anomaly map, generating the boundaries for the agricultural plot defined in the shapefile.

12. The one or more non-transitory storage media of claim 11 , wherein the agricultural plot is a part of the agricultural field.

13. The one or more non-transitory storage media of claim 11 , wherein

Assignees

• Climate Llc

Inventors

• Peshlov Boyan
• Wang Weilin

Classifications

• B64C39/02

  characterised by special use · CPC title

• B64U2101/30

  for imaging, photography or videography · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/096

  Transfer learning · CPC title

• G06N3/09

  Supervised learning · CPC title