Leaf detection and extraction system

Patent metadata

Field	Value
Publication number	US-12520772-B2
Application number	US-202318453908-A
Country	US
Kind code	B2
Filing date	Aug 22, 2023
Priority date	Aug 22, 2022
Publication date	Jan 13, 2026
Grant date	Jan 13, 2026

Abstract

Disclosed herein is a system and methods for leaf detection and extraction. The extracted leaf may be used for leaf water potential analysis. In some embodiments, the method comprises identifying the leaf from a point cloud based on an image, determining a pose of the leaf based on the point cloud, and cutting and storing the leaf based on the pose of the leaf.

First claim

What is claimed is:

1 . A computer-implemented method for collecting one or more leaves, comprising:
receiving, from a camera, an image comprising a leaf; generating, based on the image, a point cloud; identifying a portion of the point cloud associated with the leaf; determining, based on the portion of the point cloud, a pose of the leaf, wherein:
the pose comprises a position and an orientation of the leaf,
an end-effector is aligned to the leaf based on the pose of the leaf, and
a pneumatic subsystem is enabled;
enclosing the leaf in a chamber of the end-effector, wherein, after the aligning the end-effector to the leaf, the chamber of the end-effector is moved toward the leaf to enclose the leaf; cutting, via a blade of the end-effector, a stem of the leaf; and storing the leaf in the chamber of the end-effector.

2 . The method of claim 1 , wherein:
the end-effector is coupled to an arm; and the arm aligns the end-effector to the leaf and moves the end-effector toward the leaf.

3 . The method of claim 1 , wherein:
the end-effector comprises the camera; and the camera takes images of the leaves as it moves toward the leaf.

4 . The method of claim 1 , wherein:
the end-effector actuates a slider for occluding the chamber, wherein:
the slider is coupled to the blade and a piston; and
the piston actuates the slider and the blade.

5 . The method of claim 1 wherein:
the end-effector actuates a gate for occluding the chamber, wherein:
the gate is coupled to the blade and two four-bar linkages; and
the two four-bar linkages actuate the gate and the blade.

6 . The method of claim 1 , wherein:
the point cloud comprises a second portion, the second portion comprising points at distances greater than a threshold distance from the camera, and the method further comprises:
identifying the second portion; and
disregarding the second portion for the identifying the portion of the point cloud associated with the leaf.

7 . The method of claim 1 , wherein the identifying the portion of the point cloud associated with the leaf comprises:
downsampling the portion of the point cloud; and clustering the portion of the point cloud.

8 . The method of claim 1 , wherein the determining the pose of the leaf comprises:
bounding a clustered portion of the point cloud; and determining a center, dimensions, and orientation of the bounded cluster.

9 . The method of claim 1 , wherein the image comprises a second leaf, the method further comprising:
identifying a second portion of the point cloud associated with the second leaf; determining, based on the second portion of the point cloud, a second pose of the second leaf, wherein:
the second pose comprises a position and an orientation of the second leaf, and
the end-effector is aligned to the second leaf based on the second pose of the second leaf;
enclosing the second leaf in the chamber of the end-effector, wherein, after the aligning the end-effector to the second leaf, the chamber of the end-effector is moved toward the second leaf to enclose the second leaf; cutting, via the blade of the end-effector, a stem of the second leaf; and storing the second leaf in the chamber of the end-effector.

10 . The method of claim 9 , wherein the end-effector is aligned to the second leaf, suction is enabled, the end-effector is moved toward the second leaf, the stem of the second leaf is cut, and the second leaf is stored in accordance with a determination that the leaf is damaged.

11 . The method of claim 1 , wherein the pneumatic subsystem actuates suction within the end-effector using a solenoid, an air compressor, and an air tank.

12 . The method of claim 1 , further comprising:
receiving, from the camera, a second image different from the image; generating, based on the second image, a second point cloud; and identifying a portion of the second point cloud, wherein the portion of the point cloud is associated with a second leaf.

13 . The method of claim 1 , wherein the determining the pose of the leaf further comprises identifying the stem of the leaf, a tip of the leaf, and a centroid of the leaf.

14 . An end-effector for collecting one or more leaves, comprising:
a camera configured to take images of the leaves to send to the processor; a blade configured to cut the leaves; a chamber to encapsulate the leaves for cutting and storage; and one or more processors configured to cause the end-effector to perform the method of claim 1 .

15 . The end-effector of claim 14 , wherein a weight of the end-effector is less than 1.3 kg.

16 . The end-effector of claim 14 , wherein the chamber has a width of 110 mm, a height of 45 mm, and a depth of 185 mm.

17 . The end-effector of claim 14 , wherein a force for cutting the stem is 2.9-20 N, or operates at a pressure no greater than 120 psi.

18 . The end-effector of claim 14 , wherein the camera comprises a depth camera configured to provide images of the leaf to transmit to the processor for leaf detection.

19 . The end-effector of claim 14 , wherein:
the end-effector further comprises a slider for occluding the chamber, the slider is coupled to the blade and a piston, and the piston is configured to actuate the slider and the blade.

20 . The end-effector of claim 14 , wherein:
the end-effector further comprises a gate for occluding the chamber, the gate is coupled to the blade and two four-bar linkages, and the two four-bar linkages are configured to actuate the gate and the blade.

21 . A system for collecting one or more leaves, comprising:
an end-effector comprising:
a camera configured to take images of the one or more leaves to send to one or more processors;
a blade configured to cut the one or more leaves;
a chamber to encapsulate the one or more leaves for cutting and storage; and
wherein the one or more processors are configured to cause the end-effector to perform the method of claim 1 ;
a pneumatic system configured to use suction to align the one or more leaves; an arm coupled to the end-effector and configured to move the end-effector with six degrees of freedom; and a base coupled to the arm and comprising wheels.

22 . The system of claim 21 , wherein the pneumatic system comprises an air compressor, an air tank, and a solenoid.

Assignees

• Univ California

Inventors

• Karydis Konstantinos
• Campbell Merrick
• Dechemi Amel

Classifications

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T2207/30188

  Vegetation; Agriculture · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title

• G06T7/0012

  Biomedical image inspection · CPC title