Precision agricultural treatment based on growth stage in real time

Patent metadata

Field	Value
Publication number	US-11694434-B2
Application number	US-202017073244-A
Country	US
Kind code	B2
Filing date	Oct 16, 2020
Priority date	Oct 16, 2020
Publication date	Jul 4, 2023
Grant date	Jul 4, 2023

Abstract

Various embodiments of an apparatus, methods, systems and computer program products described herein are directed to an agricultural observation and treatment system and method of operation. The agricultural treatment system may determine a first real-world geo-spatial location of the treatment system. The system can receive captured images depicting real-world agricultural objects of a geographic scene. The system can associate captured images with the determined geo-spatial location of the treatment system. The treatment system can identify, from a group of mapped and indexed images, images having a second real-word geo-spatial location that is proximate with the first real-world geo-spatial location. The treatment system can compare at least a portion of the identified images with at least a portion of the captured images. The treatment system can determine a target object and emit a fluid projectile at the target object using a treatment device.

First claim

What is claimed is:

1. A method performed by an agricultural treatment system comprising one or more processors comprising hardware, one or more image sensors, and a treatment unit, the one or more processors configured to perform operations comprising:
receiving image data in a real-world agricultural scene from one or more image capture devices;
detecting one or more agricultural objects in a first image of the image data;
identifying a stage of growth associated with a first sub-object and a second sub-object of a first agricultural object;
assigning a label of the stage of growth for each of the first sub-object and the second sub-object of the first agricultural object;
determining one or more treatment parameters based on the assigned labels of the stage of growth of the first sub-object and the second sub-object of the first agricultural object;
receiving instructions for a treatment unit, of the agricultural treatment system, to receive a fluid mixture from a chemical selector to prepare a treatment;
determining a current physical location of the first sub-object based on a first pixel position in the first image or another image and a physical location of the second sub-object of the first agricultural object based on a second pixel position in the first image or the another image of the first agricultural object;
based on the determined current physical location of the first sub-object, orienting the treatment unit to target the first sub-object of the first agricultural object in the real-world agricultural scene, and activating the treatment unit to emit a first fluid projectile at a surface of the first sub-object of the first agricultural object; and
based on the determined current physical location of the second sub-object, orienting the treatment unit to target the second sub-object of the first agricultural object in the real-world agricultural scene, and activating the treatment unit to emit a second fluid projectile at a surface of the second sub-object of the first agricultural object.

2. The method of claim 1 , further comprising assigning a time stamp to each image captured of the image data including a timestamp associated with the first image.

3. The method of claim 1 , further comprising receiving localization and position data of a vehicle supporting the agricultural treatment system.

4. The method of claim 2 , further comprising generating localization and position data of the first agricultural object based on the localization and position data of the vehicle.

5. The method of claim 1 , further comprising receiving a treatment history of the first sub-object and the second sub-object of the first agricultural object detected in the first image.

6. The method of claim 5 , wherein the treatment history can be generated from one or more indexed agricultural data of a previously received and labelled image of the first sub-object and the second sub-object of the first agricultural object in the real-world agricultural scene and determining previous treatments applied to the first sub-object and the second sub-object of the first agricultural object.

7. The method of claim 6 , wherein based on the treatment history and the assigned label of the stage of growth of the first sub-object and the second sub-object of the first agricultural object, the instructions for the treatment unit can be to apply a treatment to promote growth of the first sub-object and the second sub-object of the agricultural object.

8. The method of claim 6 , wherein based on the treatment history and the assigned label of the stage of growth of the first sub-object and the second sub-object of the first agricultural object, the instructions for the treatment unit can be to apply a treatment to block growth of the first sub-object and the second sub-object of the agricultural object.

9. The method of claim 6 , wherein based on the treatment history and the assigned label of the stage of growth of the first sub-object and the second sub-object of the first agricultural object, the instructions for the treatment unit can be to apply a treatment to remove the first sub-object or the second sub-object of the agricultural object.

10. The method of claim 1 , wherein the first agricultural object detected in the agricultural scene is a specialty crop and the agricultural scene can be part of a farm, orchard, greenhouse, nursery or other geographic boundary utilized for cultivating land and harvesting crops.

11. The method of claim 1 , further comprising:
labeling a portion of the first agricultural object that requires a treatment and labeling other portions of the first agricultural object that should not be treated.

12. An agricultural treatment system, the system comprising:
one or more image sensors for obtaining imagery;
a treatment unit configured to receive instructions to select a treatment parameter and apply one or more treatments to real-world objects in a geographic boundary; and
one or more processors, memory and a non-transitory computer readable storage medium comprising hardware, the one or more processors configured to control operation of the agricultural treatment system, wherein the one or more processors is configured to perform operations comprising:
receive image data in a real-world agricultural scene from one or more image capture devices;
detect one or more agricultural objects in a first image of the image data;
identify a stage of growth associated with a first sub-object and a second sub-object of a first agricultural object;
assign a label of the stage of growth for each of the first sub-object and the second sub-object of the first agricultural object;
determine one or more treatment parameters based on the assigned labels of the stage of growth of the first sub-object and the second sub-object of the first agricultural object;
receive instructions for a treatment unit, of the agricultural treatment system, to receive a fluid mixture from a chemical selector to prepare a treatment;
determine a current physical location of the first sub-object based on a first pixel position in the first image or another image and a physical location of the second sub-object of the first agricultural object based on a second pixel position in the first image or the another image of the first agricultural object;
based on the determined current physical location of the first sub-object, orient the treatment unit to target the first sub-object of the first agricultural object in the real-world agricultural scene and activate the treatment unit to emit a first fluid projectile at a surface of the first sub-object of the first agricultural object; and
based on the determined current physical location of the second sub-object, orient the treatment unit to target the second sub-object of the first agricultural object in the real-world agricultural scene, and activating the treatment unit to emit a second fluid projectile at a surface of the second sub-object of the first agricultural object.

13. The system of claim 12 , the operations further comprising assigning a time stamp to each image captured of the image data including a timestamp associated with the first image.

14. The system of claim 12 , the operations further comprising receiving localization and position data of a vehicle supporting the agricultural treatment system.

15. The system of claim 13 , the operations further comprising generating localization and position data of the first agricultural object based on the localization and position data of the vehicle.

16. The system of claim 12 , the operations further comprising

Assignees

• Verdant Robotics Inc

Inventors

• Sibley Gabriel Thurston
• Ibarria Lorenzo
• Garner Curtis Dale
• Leger Patrick Christopher
• Michelin Andre Robert Daniel
• Hurliman Ii John Phillip
• Jirattigalochote Wisit
• Tafish Hasan

Classifications

• G05D1/2435

  Extracting 3D information · CPC title

• G06T7/74

  involving reference images or patches · CPC title

• A01B79/02

  combined with other agricultural processing, e.g. fertilising, planting · CPC title

• G06T2207/30188

  Vegetation; Agriculture · CPC title

• A01B79/005

  Precision agriculture · CPC title