Applying multiple processing schemes to target objects

Patent metadata

Field	Value
Publication number	US-11937524-B2
Application number	US-202217932566-A
Country	US
Kind code	B2
Filing date	Sep 15, 2022
Priority date	Oct 16, 2020
Publication date	Mar 26, 2024
Grant date	Mar 26, 2024

Abstract

A method includes obtaining, by the treatment system configured to implement a machine learning (ML) algorithm, one or more images of a region of an agricultural environment near the treatment system, wherein the one or more images are captured from the region of a real-world where agricultural target objects are expected to be present, determining one or more parameters for use with the ML algorithm, wherein at least one of the one or more parameters is based on one or more ML models related to identification of an agricultural object, determining a real-world target in the one or more images using the ML algorithm, wherein the ML algorithm is at least partly implemented using the one or more processors of the treatment system, and applying a treatment to the target by selectively activating the treatment mechanism based on a result of the determining the target.

First claim

The invention claimed is:

1. A method implemented by a treatment system on a moving platform having one or more processors, a storage, and a treatment mechanism, comprising:
obtaining, by the treatment system disposed on a vehicle and configured to implement a machine learning (ML) algorithm, images of a region of an agricultural environment;
applying a sequence of image processing algorithms to portions of images of the agricultural environment ingested from sensors on the moving platform, wherein the sequence of image processing algorithms includes a first image processing algorithm that performs object detection and a second image processing algorithm that performs object tracking of detected objects from the object detection;
making a decision about whether and which of the detected and tracked objects are candidates for a treatment by the treatment system; and
applying the treatment to the objects according to the decisions wherein, the sequence of image processing algorithms is applied such that upon detecting an object in a first image by the first image processing algorithm, processing of the object by the first image processing algorithm is turned off for next N frames and the object is tracked by the second image processing algorithm for the next N frames, where N is a positive integer.

2. The method of claim 1 , wherein the first image processing algorithm is a machine learning (ML) algorithm and the second image processing algorithm is a computer vision (CV) algorithm.

3. The method of claim 1 , wherein the first image processing algorithm is a first machine learning (ML) algorithm that uses a first ML model and the second image processing algorithm is a second ML algorithm that uses a second ML model.

4. The method of claim 1 , wherein the first image processing algorithm is a computer vision (CV) algorithm and the second image processing algorithm is a machine learning (ML) algorithm.

5. The method of claim 1 , wherein the first image processing algorithm is trained to detect objects that are to be included from the treatment and objects that are to be excluded from the treatment.

6. The method of claim 5 , wherein the objects that are to be excluded from the treatment are not tracked by the object tracking.

7. The method of claim 1 , wherein the object tracking comprises tracking objects that are candidates for the treatment and tracking objects that are a landmark in the agricultural environment.

8. A method implemented by a treatment system on a moving platform having one or more processors, a storage, and a treatment mechanism, comprising:
obtaining, by the treatment system disposed on a vehicle, images of a region of an agricultural environment;
determining target objects by applying a sequence of image processing algorithms to portions of images of an agricultural environment ingested from sensors on the moving platform, wherein the sequence of image processing algorithms includes a machine learning (ML) algorithm; and
applying the treatment to the objects by activating the treatment mechanism;
wherein, the sequence of image processing algorithms is applied such that upon detecting an object in a first image by a first image processing algorithm, processing of the object by the first image processing algorithm is turned off for next N frames and the object is tracked by a second image processing algorithm for the next N frames, where N is a positive integer.

9. The method of claim 8 , wherein the sequence of image processing algorithms includes multiple ML algorithms that are successively applied to the portions of images.

10. The method of claim 9 , wherein the multiple ML algorithms include an ML algorithm that is trained to detect the target objects based on one or more colors of interest.

11. The method of claim 9 , wherein the multiple ML algorithms include an ML algorithm that is configured to classify pixels of the portions of images as belonging to a target object or not belonging to any target object.

12. The method of claim 8 , wherein the sequence of image processing algorithms includes an image processing algorithm that identifies bounding boxes around pixel regions where target objects are detected.

13. The method of claim 12 , wherein the image processing algorithm is the ML algorithm.

14. The method of claim 12 , wherein the image processing algorithm is a computer vision algorithm.

15. A treatment system mountable on a moving platform, comprising:
one or more processors,
a storage configured to hold instructions or data used by the one or more processors, and
a treatment mechanism;
wherein the one or more processors are configured to:
implement a machine learning (ML) algorithm,
obtain images of a region of an agricultural environment;
apply a sequence of image processing algorithms to portions of images of the agricultural environment ingested from sensors on the moving platform, wherein the sequence of image processing algorithms includes a first image processing algorithm that performs object detection and a second image processing algorithm that performs object tracking of detected objects from the object detection;
make a decision about whether and which of the detected and tracked objects are candidates for a treatment by the treatment system; and
apply, by controlling the treatment mechanism, the treatment to the objects according to the decision;
wherein, the sequence of image processing algorithms is applied such that upon detecting an object in a first image by the first image processing algorithm, processing of the object by the first image processing algorithm is turned off for next N frames and the object is tracked by the second image processing algorithm for the next N frames, where N is a positive integer.

16. The treatment system of claim 15 , wherein the first image processing algorithm is a machine learning (ML) algorithm and the second image processing algorithm is a computer vision (CV) algorithm.

17. The treatment system of claim 15 , wherein the first image processing algorithm is a first machine learning (ML) algorithm that uses a first ML model and the second image processing algorithm is a second ML algorithm that uses a second ML model.

18. The treatment system of claim 15 , wherein the first image processing algorithm is a computer vision (CV) algorithm and the second image processing algorithm is a machine learning (ML) algorithm.

19. The treatment system of claim 15 , wherein the first image processing algorithm is trained to detect objects that are to be included from the treatment and objects that are to be excluded from the treatment.

20. The treatment system of claim 15 , wherein the object tracking comprises tracking objects that are candidates for the treatment and tracking objects that are a landmark in the agricultural environment.

Assignees

• Verdant Robotics Inc

Inventors

• Sibley Gabriel Thurston
• Ibarria Lorenzo
• Garner Curtis Dale
• Leger Patrick Christopher
• Webb Dustin James

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/096

  Transfer learning · CPC title

• G06N3/091

  Active learning · CPC title

• G06N3/09

  Supervised learning · CPC title

• G05D1/43

  Control of position or course in two dimensions [2D] · CPC title