Hybrid vision system for crop land navigation

Patent metadata

Field	Value
Publication number	US-12089110-B2
Application number	US-202318369770-A
Country	US
Kind code	B2
Filing date	Sep 18, 2023
Priority date	Oct 4, 2019
Publication date	Sep 10, 2024
Grant date	Sep 10, 2024

Abstract

Autonomous vehicles with global positioning systems are used for field inspection. A vehicle may be programmed to traverse a field, while using sensors to detect objects in the field, and then capture low-resolution images of the objects. Machine vision techniques are used with the low-resolution images to recognize the objects as crops, non-crop plant material or undefined objects. Location data is used to correlate recognized objects with digitally stored field maps to resolve whether a particular object is in a location at which crop planting is expected or not expected. Depending on whether an object in a low-resolution digital image is recognized as a crop, and whether the object is in an expected geo-location for crops, the vehicle may switch to a second image capture mode, for example, capturing a high-resolution image of the object, and/or execute a disease analysis and/or weed analysis on the images of the objects.

First claim

What is claimed is:

1. A method for determining a specific agricultural product type using autonomous mobile equipment, the method comprising:
obtaining digital image data from a digital camera on autonomous mobile equipment configured to traverse an agricultural field, the digital image data representing images of one or more plants in the agricultural field and one or more images of a specific plant to be classified, wherein the specific plant to be classified is planted at one or more field geolocations of the agricultural field and is associated with a specific agricultural product type to be determined;
receiving specific field space geolocation data representing a specific field space geolocation of the autonomous mobile equipment in the agricultural field;
comparing the specific agricultural product type to one or more pre-classified product types based on: a plant growth characteristic of the specific agricultural product type and a pre-classified plant growth characteristic corresponding to the pre-classified product type to which the specific agricultural product type matches;
comparing the specific field space geolocation to one or more pre-classified plant geolocations; and
based on the comparison of the specific agricultural product type to the one or more pre-classified product types and the comparison of the specific field space geolocation to the one or more pre-classified plant geolocations, performing at least one of:
executing a disease analysis on the one or more digital images of the specific plant associated with the specific agricultural product type;
automatically capturing at least one high-resolution digital image of the specific plant associated with the specific agricultural product type; and
executing a weed analysis on the one or more digital images of the specific plant associated with the specific agricultural product type.

2. The method of claim 1 , wherein comparing the specific field space geolocation to the one or more pre-classified plant geolocations includes determining that the specific field space geolocation does not match a pre-classified plant geolocation of the one or more pre-classified plant geolocations; and
wherein automatically capturing the at least one high-resolution digital image of the specific plant associated with the specific agricultural product type is in response to: the specific field space geolocation not matching a pre-classified plant geolocation of the one or more pre-classified plant geolocations.

3. The method of claim 1 , wherein comparing the specific agricultural product type to the one or more pre-classified product types includes determining that the specific agricultural product type matches the pre-classified product type of the one or more pre-classified product types; and
wherein comparing the specific field space geolocation to the one or more pre-classified plant geolocations includes determining that the specific field space geolocation does not match a pre-classified plant geolocation of the one or more pre-classified plant geolocations; and
wherein automatically capturing the at least one high-resolution digital image of the specific plant associated with the specific agricultural product type is in response to: the specific agricultural product type matching the pre-classified product type of the one or more pre-classified product types and the specific field space geolocation not matching a pre-classified plant geolocation of the one or more pre-classified plant geolocations.

4. The method of claim 3 , further comprising:
determining whether one or more attributes of the specific plant in the at least one high-resolution digital image are consistent with expected growth for the specific agricultural product type with which the specific plant is associated; and
in response to determining that the one or more attributes of the specific plant in the at least one high-resolution digital image are not consistent with expected growth for the specific agricultural product type with which the specific plant is associated, executing a weed analysis on the at least one high-resolution digital image of the specific plant associated with the specific agricultural product type.

5. The method of claim 1 , wherein comparing the specific agricultural product type to the one or more pre-classified product types includes determining that the specific agricultural product type matches the pre-classified product type of the one or more pre-classified product types; and
wherein comparing the specific field space geolocation to the one or more pre-classified plant geolocations includes determining that the specific field space geolocation does not match a pre-classified plant geolocation of the one or more pre-classified plant geolocations; and
wherein executing the weed analysis on the one or more digital images of the specific plant associated with the specific agricultural product type is in response to: the specific agricultural product type matching the pre-classified product type of the one or more pre-classified product types and the specific field space geolocation not matching a pre-classified plant geolocation of the one or more pre-classified plant geolocations.

6. The method of claim 5 , wherein executing the weed analysis comprises generating, via a trained machine learning classifier, a classification output specifying the digital image data representing the one or more images of the specific plant as a particular weed.

7. The method of claim 1 , wherein comparing the specific agricultural product type to the one or more pre-classified product types includes determining that the specific agricultural product type matches the pre-classified product type of the one or more pre-classified product types; and
wherein comparing the specific field space geolocation to the one or more pre-classified plant geolocations includes determining that the specific field space geolocation matches a pre-classified plant geolocation of the one or more pre-classified plant geolocations; and
wherein executing the disease analysis on the one or more digital images of the specific plant associated with the specific agricultural product type is in response to: the specific field space geolocation not matching a pre-classified plant geolocation of the one or more pre-classified plant geolocations.

8. The method of claim 7 , wherein executing the disease analysis comprises generating, via a trained machine learning classifier, a classification output specifying the digital image data representing the one or more images of the specific plant as a diseased plant candidate.

9. The method of claim 1 , wherein the plant growth characteristic of the specific agricultural product type is obtained from one or more sensor devices, each of the one or more sensor devices being located at a location in or near the agricultural field.

10. The method of claim 1 , further comprising determining a plant location match between the specific field space geolocation and one or more pre-classified agricultural product geolocations based on a digitally stored plot map of the agricultural field, the plot map including a geolocation of the specific agricultural product.

11. A non-transitory computer-readable storage medium comprising executable instructions for determining a specific agricultural product type using autonomous mobile equipment, which when executed by at least one processor, cause the at least one processor to:
obtain digital image data from a digital camera on autonomous mobile equipment configured to traverse an agricultural field, the digital image data representing images of one or more plants in the agricultural field and one or more images of

Assignees

• Climate Llc

Inventors

• Young Mark

Classifications

• G05D1/249

  from positioning sensors located off-board the vehicle, e.g. from cameras · CPC title

• G06V20/10

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title

• G06V10/87

  using selection of the recognition techniques, e.g. of a classifier in a multiple classifier system · CPC title

• G06F18/2413

  based on distances to training or reference patterns · CPC title

• G06V30/2504

  Coarse or fine approaches, e.g. resolution of ambiguities or multiscale approaches · CPC title