Detecting and treating a target from a moving platform

Patent metadata

Field	Value
Publication number	US-11751559-B2
Application number	US-202218061860-A
Country	US
Kind code	B2
Filing date	Dec 5, 2022
Priority date	Oct 20, 2021
Publication date	Sep 12, 2023
Grant date	Sep 12, 2023

Abstract

A method includes receiving, by the treatment system, during operation in an agricultural environment, one or more images comprising one or more agricultural objects in the agricultural environment, identifying, in real-time, one or more objects of interest from the one or more agricultural objects by analyzing the one or more images, wherein the analyzing results in a first object being identified as belonging to one or more target objects and a second object being identified as not belonging to the one or more target objects, logging one or more results of the identification of each of the one or more objects of interest and a corresponding treatment decision; and activating the treatment mechanism to treat the one or more target objects.

First claim

What is claimed is:

1. A method performed by a treatment system on a moving platform, the treatment system having one or more processors, a storage, and a treatment mechanism, comprising:
receiving, by the treatment system, during operation in an agricultural environment, one or more sensor readings comprising one or more agricultural objects in the agricultural environment;
identifying, one or more objects of interest from the one or more agricultural objects by analyzing the one or more sensor readings;
determining a first target object of the one or more objects of interest for treatment;
determining a predicted position of the first target object in the agricultural environment relative to the treatment mechanism wherein the predicted position of the first object is determined by tracking the first target object as the treatment mechanism traverses the agricultural environment;
activating the treatment mechanism to treat the first target object; and
detecting a treatment of the first target object;
wherein the detecting the treatment of the first target object includes detecting a fluid, the fluid comprising a fluid projectile or a fluid spray, emitted from the treatment mechanism towards the first target object and determining whether the fluid will intercept the first target object, determining whether the fluid has intercepted the first target object, or a combination thereof;
wherein the detecting the fluid comprises performing homography estimation based on the one or more sensor readings captured on the moving platform.

2. The method of claim 1 , wherein the analyzing the one or more sensor readings comprises analyzing the one or more sensor readings using a machine learning (ML) algorithm that uses one or more ML models configured to identify the one or more agricultural objects.

3. The method of claim 2 , wherein the ML algorithm is generated at least in part based on transfer learning from prior ML models or prior versions of the one or more ML models configured to detect plant objects, or landmarks, or both, of a different agricultural environment than the agricultural environment.

4. The method of claim 1 , further comprising orienting the treatment mechanism to point at the predicted position as the treatment system tracks the first target object.

5. The method of claim 1 , further comprising determining a pose estimation of the treatment system by determining a global pose using inputs from sensors rigidly connected to the moving platform configured to receive sensor readings of a world environment causing determination of a location and an orientation of the moving platform, determining a local pose from sensors configured to sense a local environment of the treatment system causing determination of a localization and an orientation of the treatment system, or a combination thereof.

6. The method of claim 1 , wherein tracking the first target object is performed for a period of time subsequent to the determining the first target object for treatment.

7. The method of claim 1 , wherein the one or more sensor readings are one or more images captured by one or more image capture devices.

8. The method of claim 7 , wherein the one or more objects of interests, including the first target object, is identified in a first image of the one or more images and the first target object is tracked via multiple subsequent images of the one or more images.

9. The method of claim 1 , wherein the activating the treatment mechanism includes emitting a laser or emitting a fluid projectile or fluid spray or configuring an end effector to physically interact with the first target object.

10. The method of claim 1 , wherein detecting the treatment of the first target object includes detecting a fluid, the fluid comprising a fluid projectile or a fluid spray, emitted from the treatment mechanism towards the first target object and determining whether the fluid will intercept the first target object, determining whether the fluid has intercepted the first target object, or a combination thereof.

11. The method of claim 10 , wherein detecting the fluid comprises performing image differencing of multiple images.

12. The method of claim 1 , wherein detecting the treatment of the first target object includes detecting a pattern, the pattern comprising a spot pattern, a splat pattern, or a splash pattern, on or near a portion of ground of the agricultural environment, and determining whether the pattern is identified to be on or near the first target object, determining whether the first target object is identified to be on or near the pattern, or a combination thereof.

13. The method of claim 12 , wherein detecting the pattern comprises performing image differencing of multiple images.

14. The method of claim 1 , further comprising logging the detected treatment and using a result of the logged detected treatment to generate performance metrics of the treatment system, generate predicted characteristics of the first target object including a future yield, a future size, a future health, a future disease, or a combination thereof.

15. The method of claim 1 , further comprising logging one or more results of identification of each of the one or more objects of interest.

16. A method performed by a treatment system on a moving platform, the treatment system having one or more processors, a storage, and a treatment mechanism, comprising:
receiving, by the treatment system, during operation in an agricultural environment, one or more sensor readings comprising one or more agricultural objects in the agricultural environment;
identifying, one or more objects of interest from the one or more agricultural objects by analyzing the one or more sensor readings;
determining a first target object of the one or more objects of interest for treatment;
determining a predicted position of the first target object in the agricultural environment relative to the treatment mechanism wherein the predicted position of the first object is determined by tracking the first target object as the treatment mechanism traverses the agricultural environment;
activating the treatment mechanism to treat the first target object; and
detecting a treatment of the first target object;
wherein the detecting the treatment of the first target object includes detecting a pattern, the pattern comprising a spot pattern, a splat pattern, or a splash pattern, on or near a portion of ground of the agricultural environment, and determining whether the pattern is identified to be on or near the first target object, determining whether the first target object is identified to be on or near the pattern, or a combination thereof;
wherein the detecting the pattern comprises performing homography estimation based on the one or more sensor readings captured on the moving platform.

17. The method of claim 16 , wherein the analyzing the one or more sensor readings comprises analyzing the one or more sensor readings using a machine learning (ML) algorithm that uses one or more ML models configured to identify the one or more agricultural objects.

18. The method of claim 17 , wherein the ML algorithm is generated at least in part based on transfer learning from prior ML models or prior versions of the one or more ML models configured to detect plant objects, or landmarks, or both, of a different agricultural environment than the agricultural environment.

19. The method of claim 16 , further comprising orienting the treatment mechanism to point at the predicted position as the treatment system tracks

Assignees

• Verdant Robotics Inc

Inventors

• Sibley Gabriel Thurston
• Ibarria Lorenzo
• Garner Curtis Dale
• Leger Patrick Christopher

Classifications

• A01M7/0089 Primary

  Regulating or controlling systems (the delivery being related to the movement of a vehicle B05B9/06) · CPC title

• A01M9/0092

  Regulating or controlling systems (the delivery being related to the movement of a vehicle B05B9/06) · CPC title

• A01M21/00

  Apparatus for the destruction of unwanted vegetation, e.g. weeds (control of undesirable vegetation on roads or permanent ways of railways E01H11/00) · CPC title

• A01C21/007

  Determining fertilization requirements · CPC title

• A01B79/005

  Precision agriculture · CPC title