Autonomous agricultural treatment system using map based targeting of agricultural objects

Patent metadata

Field	Value
Publication number	US-11076589-B1
Application number	US-202017073236-A
Country	US
Kind code	B1
Filing date	Oct 16, 2020
Priority date	Oct 16, 2020
Publication date	Aug 3, 2021
Grant date	Aug 3, 2021

Abstract

Various embodiments of an apparatus, methods, systems and computer program products described herein are directed to an agricultural observation and treatment system and method of operation. The agricultural treatment system may determine a first real-world geo-spatial location of the treatment system. The system can receive captured images depicting real-world agricultural objects of a geographic scene. The system can associate captured images with the determined geo-spatial location of the treatment system. The treatment system can identify, from a group of mapped and indexed images, images having a second real-word geo-spatial location that is proximate with the first real-world geo-spatial location. The treatment system can compare at least a portion of the identified images with at least a portion of the captured images. The treatment system can determine a target object and emit a fluid projectile at the target object using a treatment device.

First claim

What is claimed is:

1. A method performed by an agricultural treatment system comprising one or more processors comprising hardware, one or more image sensors, and a treatment unit, the one or more processors configured to perform operations comprising:
storing a group of images on a data storage device, the group of images depicting views of real-world agricultural objects, the one or more stored images having associated real-world geo-spatial locations, and the one or more stored images including pre-identified agricultural objects;
determining a first real-world geo-spatial location of the agricultural treatment system;
receiving one or more captured images depicting real-world agricultural objects of a geographic scene;
associating the one or more captured images with the determined geo-spatial location of the agricultural treatment system;
identifying, from the group of one or more stored images, one or more images having a second real-word geo-spatial location that is proximate with the first real-world geo-spatial location, wherein the one or more captured images are obtained at a time later than when the group of images are stored on the storage device;
comparing at least a portion of the identified images with the one or more captured images;
determining a target object for treatment based on the comparing of the one or more captured images with the portions of the one or more identified images, the target object being determined where a portion of a captured image corresponds to a portion of an image from the identified one or more images; and
emitting a fluid projectile at the target object using the treatment unit.

2. The method of claim 1 , wherein the group of images comprise key frame images and sub-key frame images, the key frame images depicting agriculture objects of the geographical scene, and the sub-key frame images depicting a portion of a key frame image.

3. The method of claim 2 , wherein comparing at least a portion of the identified images comprises:
comparing pixels of the sub-key frame image to a portion of one of the captured images;
determining a confidence level of whether the sub-key frame image matches the portion of the captured image; and
identifying a match where the determined confidence level meets or exceeds a threshold value.

4. The method of claim 2 , wherein the key frame images were previously obtained by the one or more image sensors of the agricultural treatment system, and the captured images are obtained by the same one or more image sensors at a time subsequent to when the key frame images were taken.

5. The method of claim 1 , wherein comparing at least a portion of the identified images comprises:
identifying a landmark object in a captured image; and
determining a portion of the landmark object in the capture image matches a portion of an image from the group of images.

6. The method of claim 5 , wherein the target objects are real-world objects that are intended to be treated with the fluid projectile, and the landmark objects are real-world objects that aid in determining the location of a target object.

7. The method of claim 1 , wherein the one or more captured images are obtained by one or more image sensors and the real-world agricultural objects are located within a distance of 0.5 foot to 100 feet from the camera capturing the images.

8. A non-transitory computer storage medium comprising instructions that when executed by one or more process included of an agricultural treatment system, cause the agricultural treatment system to perform the operations comprising:
storing a group of images on a data storage device, the group of images depicting views of real-world agricultural objects, the one or more stored images having associated real-world geo-spatial locations, and the one or more stored images including pre-identified agricultural objects;
determining a first real-world geo-spatial location of the agricultural treatment system;
receiving one or more captured images depicting real-world agricultural objects of a geographic scene;
associating the one or more captured images with the determined geo-spatial location of the agricultural treatment system;
identifying, from the group of one or more stored images, one or more images having a second real-word geo-spatial location that is proximate with the first real-world geo-spatial location, wherein the one or more captured images are obtained at a time later than when the group of images are stored on the storage device;
comparing at least a portion of the identified images with the one or more captured images;
determining a target object for treatment based on the comparing of the one or more captured images with the portions of the one or more identified images, the target object being determined where a portion of a captured image corresponds with a portion of an image from the identified one or more images; and
emitting a fluid projectile at the target object using a treatment unit of the agricultural treatment system.

9. The non-transitory computer storage medium of claim 8 , wherein the group of images comprise key frame images and sub-key frame images, the key frame images depicting agriculture objects of the geographical scene, and the sub-key frame images depicting a portion of a key frame image.

10. The non-transitory computer storage medium of claim 9 , wherein comparing at least a portion of the identified images comprises:
comparing pixels of the sub-key frame image to a portion of one of the captured images;
determining a confidence level of whether the sub-key frame image matches the portion of the captured image; and
identifying a match where the determined confidence level meets or exceeds a predetermined confidence level threshold value.

11. The non-transitory computer storage medium of claim 9 , wherein the key frame images were previously obtained by the one or more image sensors of the agricultural treatment system, and the captured images are obtained by the same one or more image sensors at a time subsequent to when the key frame images were taken.

12. The non-transitory computer storage medium of claim 8 , wherein comparing at least a portion of the identified images comprises:
identifying a landmark object in a captured image; and
determining a portion of the landmark object in the capture image matches a portion of an image from the group of images.

13. The non-transitory computer storage medium of claim 12 , wherein the target objects are real-world objects that are intended to be treated with the fluid projectile, and the landmark objects are real-world objects that aid in determining the location of a target object.

14. The non-transitory computer storage medium of claim 8 , wherein the one or more captured images are obtained by one or more image sensors and the real-world agricultural objects are located within a distance of 0.5 foot to 100 feet from the camera capturing the images.

15. An agricultural treatment system, the system comprising:
one or more image sensors for obtaining imagery;
a treatment unit mounted to a gimble, the treatment unit configured to emit a fluid projectile;
one or more tanks, wherein the tanks provide a container for a fluid;
an electronically controlled pump fluidly connected to the one or more tanks and the treatment unit; and
one or more processors, memory, and a non-transitory computer readable storage medium comprising hardware, the one or more processors configured to control operation of the treatment unit, and control actuation of the electronically controlled pump and orientation of the treatment unit for

Assignees

• Verdant Robotics Inc

Inventors

• Sibley Gabriel Thurston
• Ibarria Lorenzo
• Garner Curtis Dale
• Leger Patrick Christopher
• Michelin Andre Robert Daniel
• Hurliman Ii John Phillip
• Jirattigalochote Wisit
• Tafish Hasan

Classifications

• G06F18/2413

  based on distances to training or reference patterns · CPC title

• G06V10/764 Primary

  using classification, e.g. of video objects · CPC title

• G06V20/188 Primary

  Vegetation · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06V20/10

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title