Crop biometrics detection

Patent metadata

Field	Value
Publication number	US-11188752-B2
Application number	US-201916296856-A
Country	US
Kind code	B2
Filing date	Mar 8, 2019
Priority date	Mar 8, 2018
Publication date	Nov 30, 2021
Grant date	Nov 30, 2021

Abstract

Systems, techniques, and devices for detecting plant biometrics, for example, plants in a crop field. An imaging device of an unmanned vehicle may be used to generate a plurality of images of the plants, and the plurality of images may be used to generate a 3D model of the plants. The 3D model may define locations and orientations of leaves and stems of plants. The 3D model may be used to determine at least one biometric parameter of at least one plant in the crop. Such detection of plant biometrics may facilitate the automation of crop monitoring and treatment.

First claim

What is claimed is:

1. A system for detecting crop biometrics, the system comprising:
a crop modeling device communicatively coupled to an imaging device, wherein the crop modeling device comprises processing circuitry configured to:
receive, from the imaging device, a plurality of images of a crop of a plurality of plants in a field, wherein the plurality of plants in the field include at least one of overlapping or occluding leaves,
generate, based on the plurality of images, a three-dimensional (3D) model of the plurality of plants, wherein the 3D model defines locations and orientations of leaves and stems of respective plants of the plurality of plants, and
determine, based on the 3D model, at least one of a plant height, a leaf count, a leaf angle, or an inter-nodal distance of a respective plant of the plurality of plants in the field, the respective plant including the at least one of the overlapping or the occluding leaves,
wherein the processor is configured to generate the 3D model of the plurality of plants by at least:
generating an initial point cloud reconstruction of the plurality of plants in the field based on the plurality of images,
simulating respective routes of a plurality of randomly selected points of the initial point cloud by:
identifying a ground plane within a 3D space of the initial point cloud reconstruction,
simulating, for each of a plurality of randomly selected points within the initial point cloud, point-by-point movement from an initial position of the selected point within the initial point cloud reconstruction towards the identified ground plane based on a height of respective points within the initial point cloud, and
determining the respective routes of the plurality of randomly selected points based on the simulated point by point movement,

detecting respective stems of the plurality of plants based on the simulated respective routes, and
following the detection of the respective stems of the plurality of plants, separating the at least one of overlapping or occluding leaves of the plurality of plants in the field by segmenting the leaves connected to the respective stems using the initial point cloud reconstruction.

2. The system of claim 1 , further comprising an unmanned aerial vehicle including the at least one imaging device.

3. The system of claim 1 , wherein the processor is configured to determine the at least one of the plant height, the leaf count, the leaf angle, or the inter-nodal distance of the respective plant including the at least one of the overlapping or the occluding leaves by at least:
locating, following the separation of the at least one of overlapping or occluding leaves, a respective stem and a respective uppermost leaf of each respective plant,
determining a separation between the base of the respective stem and the respective uppermost leaf, and
determining, based on the separation between the base of the respective stem and the respective uppermost leaf, the plant height of the respective plant.

4. The system of claim 3 , wherein the processor is configured to determine the separation between the base of the respective stem and the respective uppermost leaf by at least determining an average lowest point for a predetermined plurality of plants adjacent to and comprising the respective plant.

5. The system of claim 1 , wherein the processor is configured to determine the at least one of the plant height, the leaf count, the leaf angle, or the inter-nodal distance of the respective plant including the at least one of the overlapping or the occluding leaves by at least:
locating, following the separation of the at least one of overlapping or occluding leaves, a respective stem and a respective uppermost leaf of each respective plant,
determining, based on the segmentation, a plurality of leaf-stem nodes of each respective plant, and
determining, based on the leaf-stem nodes, at least one of the leaf count, the leaf angle, or the inter-nodal distance of each respective plant.

6. The system of claim 5 , wherein the processor generates the initial point cloud reconstruction of the plurality of plants in the field based on the plurality of images by at least:
generating a reduced point cloud reconstruction by reduce the number of points in the initial point cloud reconstruction by k-means clustering.

7. The system of claim 1 , wherein the processor is configured to:
receive, from the imaging device, a plurality of images of a leaf of a plant in a field,
generate, based on the plurality of images, a point cloud model of the leaf,
generate, based on the point cloud model, a self-organized map representing a surface of the leaf, and
determine, based on the self-organized map, a total surface area of the leaf.

8. The system of claim 7 , wherein the plurality of images includes images taken at different locations and different orientations about the leaf.

9. The system of claim 7 , wherein the self-organized map is defined by four-sided polygons.

10. A method for detecting crop biometrics, the method comprising:
receiving, by processing circuitry and from an imaging device, a plurality of images of a crop of a plurality of plants in a field, wherein the plurality of plants in the field include at least one of overlapping or occluding leaves;
generating, by the processing circuitry, based on the plurality of images, a three-dimensional (3D) model of the plurality of plants, wherein the 3D model defines locations and orientations of leaves and stems of respective plants of the plurality of plants; and
determining, by the processing circuitry and based on the 3D model, at least one of a plant height, a leaf count, a leaf angle, or an inter-nodal distance of a respective plant of the plurality of plants in the field, the respective plant including the at least one of the overlapping or the occluding leaves,
wherein generating the 3D model of the plurality plants comprises:
generating an initial point cloud reconstruction of the plurality of plants in the field based on the plurality of images,
simulating respective routes of a plurality of randomly selected points of the initial point cloud by:
identifying a ground plane within a 3D space of the initial point cloud reconstruction,
simulating, for each of a plurality of randomly selected points within the initial point cloud, point-by-point movement from an initial position of the selected point within the initial point cloud reconstruction towards the identified ground plane based on a height of respective points within the initial point cloud, and
determining the respective routes of the plurality of randomly selected points based on the simulated point by point movement,

detecting respective stems of the plurality of plants based on the simulated respective routes, and
following the detection of the respective stems of the plurality of plants, separating the at least one of overlapping or occluding leaves of the plurality of plants in the field by segmenting the leaves connected to the respective stems using the initial point cloud reconstruction.

11. The method of claim 10 , wherein the imaging device is part of an unmanned vehicle, and wherein the unmanned vehicle comprises an unmanned aerial vehicle.

12. The method of claim 10 , wherein determining, based on the 3D model, the at least one of the plant height, the leaf count, the leaf angle, or the inter-nodal distance of the respective plant including the at least one of the overlapping or the occluding leaves comprises:
locating, following the separation of the at least one of overlapping or occluding leaves, by the processor, a respective stem and a respective uppermost l

Assignees

• Univ Minnesota

Inventors

• Papanikolopoulos Nikolaos
• Morellas Vassilios
• Zermas Dimitris
• Mulla David
• Bazakos Mike

Classifications

• G06Q50/02 Primary

  Agriculture; Fishing; Forestry; Mining · CPC title

• G06V10/763

  Non-hierarchical techniques, e.g. based on statistics of modelling distributions · CPC title

• G06V20/17

  taken from planes or by drones · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06V20/188

  Vegetation · CPC title