Autonomous agricultural treatment system using map based targeting of agricultural objects

Patent metadata

Field	Value
Publication number	US-11751558-B2
Application number	US-202117392202-A
Country	US
Kind code	B2
Filing date	Aug 2, 2021
Priority date	Oct 16, 2020
Publication date	Sep 12, 2023
Grant date	Sep 12, 2023

Abstract

Various embodiments of an apparatus, methods, systems and computer program products described herein are directed to an agricultural observation and treatment system and method of operation. The agricultural treatment system may determine a first real-world geo-spatial location of the treatment system. The system can receive captured images depicting real-world agricultural objects of a geographic scene. The system can associate captured images with the determined geo-spatial location of the treatment system. The treatment system can identify, from a group of mapped and indexed images, images having a second real-word geo-spatial location that is proximate with the first real-world geo-spatial location. The treatment system can compare at least a portion of the identified images with at least a portion of the captured images. The treatment system can determine a target object and emit a fluid projectile at the target object using a treatment device.

First claim

What is claimed is:

1. A method performed by an agricultural treatment system comprising one or more processors comprising hardware, one or more sensors, and a treatment unit, the one or more processors configured to perform operations comprising:
receiving sensor data of a plurality of pre identified agricultural objects including each of the pre identified agricultural objects' real-world location;
determining a first real-world location of the agricultural treatment system;
receiving captured sensor data depicting agricultural objects;
selecting one or more pre identified agricultural objects, wherein the real-world locations of the selected pre identified agricultural objects are proximate to the first real-world location;
comparing at least a portion of the selected pre identified agricultural objects with the captured sensor data;
identifying a target object from the comparing of at least one selected pre identified agricultural object with at least a portion of the captured sensor data.

2. The method of claim 1 , wherein the sensor data of a plurality of pre identified agricultural objects comprise a plurality of images depicting agricultural objects.

3. The method of claim 2 , wherein comparing at least a portion of the selected pre identified agricultural objects comprises:
comparing pixels of the images depicting agricultural objects to a portion of the captured sensor data comprising captured images; and
identifying a match where a determined confidence level meets or exceeds a threshold value.

4. The method of claim 1 , further comprising associating the one or more captured sensor data with the first real-world geo spatial location of the agricultural treatment system.

5. The method of claim 1 , wherein comparing at least a portion of the selected pre identified agricultural objects comprises:
identifying a landmark object in the plurality of pre identified agricultural objects; and
matching a portion of the landmark object in the pre identified agricultural objects to a portion of the captured sensor data.

6. The method of claim 5 , wherein the target object is a real-world object intended to be treated with a fluid projectile or light source.

7. The method of claim 1 , wherein determining a first real-world location of the agricultural treatment system comprises determine a first pose of the treatment system.

8. The method of claim 7 , wherein the first pose is determined by performing SLAM.

9. The method of claim 8 , wherein the first pose is determined with one or more stereo cameras.

10. The method of claim 1 , wherein the sensor data of a plurality of pre identified agricultural objects comprises portions of labelled 2D images, portions of labelled 3D images, portions of 2D models, portions of 3D models, or a combination thereof depicting views of one or more unique agricultural objects in the real-world.

11. A non-transitory computer storage medium comprising instructions that when executed by one or more processors included of an agricultural treatment system, cause the agricultural treatment system to perform the operations comprising:
receiving sensor data of a plurality of pre identified agricultural objects including each of the pre identified agricultural objects' real-world location;
determining a first real-world location of the agricultural treatment system;
receiving captured sensor data depicting agricultural objects;
selecting one or more pre identified agricultural objects, wherein the real-world locations of the selected pre identified agricultural objects are proximate to the first real-world location;
comparing at least a portion of the selected pre identified agricultural objects with the captured sensor data;
identifying a target object from the comparing of at least one selected pre identified agricultural object with at least a portion of the captured sensor data.

12. The non-transitory computer storage medium of claim 11 , wherein the sensor data of a plurality of pre identified agricultural objects comprise a plurality of images depicting agricultural objects.

13. The non-transitory computer storage medium of claim 12 , wherein comparing at least a portion of the selected pre identified agricultural objects comprises:
comparing pixels of the images depicting agricultural objects to a portion of the captured sensor data comprising captured images; and
identifying a match where a determined confidence level meets or exceeds a threshold value.

14. The non-transitory computer storage medium of claim 11 , further comprising associating the one or more captured sensor data with the first real-world geo spatial location of the agricultural treatment system.

15. The non-transitory computer storage medium of claim 11 , wherein comparing at least a portion of the selected pre identified agricultural objects comprises:
identifying a landmark object in the plurality of pre identified agricultural objects; and
matching a portion of the landmark object in the pre identified agricultural objects to a portion of the captured sensor data.

16. The non-transitory computer storage medium of claim 15 , wherein the target object is a real-world object intended to be treated with a fluid projectile or light source.

17. The non-transitory computer storage medium of claim 11 , wherein determining a first real-world location of the agricultural treatment system comprises determine a first pose of the treatment system.

18. The non-transitory computer storage medium of claim 17 , wherein the first pose is determined by performing SLAM.

19. The non-transitory computer storage medium of claim 11 , wherein the sensor data of a plurality of pre identified agricultural objects comprises portions of labelled 2D images, portions of labelled 3D images, portions of 2D models, portions of 3D models, or a combination thereof depicting views of one or more unique agricultural objects in the real-world.

20. A method performed by an agricultural treatment system comprising one or more processors comprising hardware, one or more sensors, and a treatment unit, the one or more processors configured to perform operations comprising:
receiving sensor data of a plurality of pre identified agricultural objects including each of the pre identified agricultural objects' real-world location;
determining a first real-world location of the agricultural treatment system;
receiving captured sensor data depicting agricultural objects;
selecting one or more pre identified agricultural objects, wherein the real-world locations of the selected pre identified agricultural objects are proximate to the first real-world location;
comparing at least a portion of the selected pre identified agricultural objects with the captured sensor data;
identifying a target object from the comparing of at least one selected pre identified agricultural object with at least a portion of the captured sensor data;
activating the treatment unit comprising emitting a chemical fluid projectile or shining a light source at the target object.

Assignees

• Verdant Robotics Inc

Inventors

• Sibley Gabriel Thurston
• Ibarria Lorenzo
• Garner Curtis Dale
• Leger Patrick Christopher
• Michelin Andre Robert Daniel
• Hurliman Ii John Phillip
• Jirattigalochote Wisit
• Tafish Hasan

Classifications

• G06V20/188 Primary

  Vegetation · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06V10/764 Primary

  using classification, e.g. of video objects · CPC title

• A01M7/0089 Primary

  Regulating or controlling systems (the delivery being related to the movement of a vehicle B05B9/06) · CPC title

• A01M7/0042

  Field sprayers, e.g. self-propelled, drawn or tractor-mounted · CPC title