Crop type classification in images

Patent metadata

Field	Value
Publication number	US-10909368-B2
Application number	US-201816218305-A
Country	US
Kind code	B2
Filing date	Dec 12, 2018
Priority date	Jan 23, 2018
Publication date	Feb 2, 2021
Grant date	Feb 2, 2021

Abstract

In embodiments, obtaining a plurality of image sets associated with a geographical region and a time period, wherein each image set of the plurality of image sets comprises multi-spectral and time series images that depict a respective particular portion of the geographical region during the time period, and predicting one or more crop types growing in each of particular locations within the particular portion of the geographical region associated with an image set of the plurality of image sets. Determining a crop type classification for each of the particular locations based on the predicted one or more crop types for the respective particular locations, and generating a crop indicative image comprising at least one image of the multi-spectral and time series images of the image set overlaid with indications of the crop type classification determined for the respective particular locations.

First claim

What is claimed is:

1. A method comprising:
obtaining a plurality of image sets associated with a geographical region and a time period, wherein each image set of the plurality of image sets comprises multi-spectral and time series images that depict a respective particular portion of the geographical region during the time period;
predicting one or more crop types growing in each of particular locations within the particular portion of the geographical region associated with an image set of the plurality of image sets;
determining a crop type classification for each of the particular locations based on the predicted one or more crop types for the respective particular locations; and
generating a crop indicative image comprising at least one image of the multi-spectral and time series images of the image set overlaid with indications of the crop type classification determined for the respective particular locations;
wherein determining the crop type classification for each of the particular locations comprises:
in response to determining that the crop types predicted for the respective particular location include a dominant majority predicted crop type, selecting the dominant majority predicted crop type as the crop type classification; and
in response to determining that the crop types predicted for the respective particular location does not include a dominant majority predicted crop type:
splitting the respective particular location into a plurality of sub-particular locations; and

classifying each respective sub-particular location as a respective crop type of the crop types predicted for the particular location.

2. The method of claim 1 , wherein predicting the one or more crop types growing in each of the particular locations comprises:
predicting presence of a crop at the particular locations;
determining crop boundary locations within the particular portion of the geographical region based on the predicted presence of the crop at the particular locations; and
predicting the one or more crop types growing within each of the determined crop boundary locations.

3. The method of claim 1 , further comprising estimating a crop yield for each of the particular locations based on the crop type classification determined for the respective particular locations.

4. The method of claim 1 , further comprising determining crop management practices for each of the particular locations based on the crop type classification determined for the respective particular locations.

5. The method of claim 1 , wherein determining the crop type classification for each of the particular locations comprises determining the crop type classification to a sub-meter ground resolution for each of the particular locations.

6. The method of claim 1 , wherein predicting the one or more crop types growing in each of the particular locations comprises applying the image set to one or more machine learning systems or a convolutional neural network (CNN).

7. The method of claim 6 , wherein the one or more machine learning systems or CNN is configured to predict the one or more crop types growing in each of the particular locations after supervised training on ground truth data.

8. The method of claim 7 , wherein the ground truth data comprises one or more of government crop data, publicly available crop data, images with crop areas identified at low ground resolution, images with crop types identified at low ground resolution, images with manually identified crop boundaries, images with manually identified crop boundaries and crop types, crop survey data, sampled crop data, and farmer reports.

9. The method of claim 1 , wherein predicting the one or more crop types growing in each of the particular locations comprises, for each of the particular locations, analyzing the time series images for changes over time of pixels associated with the respective particular locations, wherein a particular change pattern of the pixels is associated with at least one crop type.

10. The method of claim 1 , further comprising:
causing to display the crop indicative image on a device accessible by a user; and
receiving a modification, from the user, of a particular indication from among the indications of the crop type classification determined for the respective particular locations, wherein the modification comprises a manual re-classification of the crop type for the particular location associated with the particular indication.

11. One or more non-transitory computer-readable storage media comprising a plurality of instructions to cause an apparatus, in response to execution by one or more processors of the apparatus, to:
obtain a plurality of image sets associated with a geographical region and a time period, wherein each image set of the plurality of image sets comprises multi-spectral and time series images that depict a respective particular portion of the geographical region during the time period;
predict one or more crop types growing in each of particular locations within the particular portion of the geographical region associated with an image set of the plurality of image sets;
determine a crop type classification for each of the particular locations based on the predicted one or more crop types for the respective particular locations; and
generate a crop indicative image comprising at least one image of the multi-spectral and time series images of the image set overlaid with indications of the crop type classification determined for the respective particular locations;
wherein to predict the one or more crop types growing in each of the particular locations comprises to apply the image set to one or more machine learning systems, wherein the one or more machine learning systems include a convolutional neural network (CNN); and
wherein the one or more machine learning systems are configured to predict the one or more crop types rowing in each of the particular locations after supervised training on ground truth data.

12. The computer-readable storage medium of claim 11 , wherein to predict the one or more crop types growing in each of the particular locations comprises to:
predict presence of a crop at the particular locations;
determine crop boundary locations within the particular portion of the geographical region based on the predicted presence of the crop at the particular locations; and
predict the one or more crop types growing within each of the determined crop boundary locations.

13. The computer-readable storage medium of claim 11 , wherein to determine the crop type classification for each of the particular locations comprises, for each of the particular locations, to select a dominant majority predicted crop type from among the crop types predicted for the respective particular locations, wherein the dominant majority predicted crop type is the crop type classification.

14. The computer-readable storage medium of claim 11 , wherein to determine the crop type classification for each of the particular locations comprises:
for each of the particular locations, if the dominant majority predicted crop type is absent, to split the respective particular location into a plurality of sub-particular locations and classify each of the respective sub-particular locations of the plurality of sub-particular locations as a respective crop type of the crop types predicted for the particular location.

15. The computer-readable storage medium of claim 11 , wherein to determine the crop type classification for each of the particular locations comprises to determine the crop type classification to a sub-meter ground resolution for each

Assignees

• X Dev Llc

Inventors

• Guo Cheng-En
• Yang Jie
• Grant Elliott

Classifications

• G06V20/17

  taken from planes or by drones · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06V20/188

  Vegetation · CPC title

• G06N3/08 Primary

  Learning methods · CPC title