Crop models and biometrics

Patent metadata

Field	Value
Publication number	US-11275941-B2
Application number	US-201916296941-A
Country	US
Kind code	B2
Filing date	Mar 8, 2019
Priority date	Mar 8, 2018
Publication date	Mar 15, 2022
Grant date	Mar 15, 2022

Abstract

Systems, techniques, and devices for detecting plant biometrics, for example, plants in a crop field. An imaging device of an unmanned vehicle may be used to generate a plurality of images of the plants, and the plurality of images may be used to generate a 3D model of the plants. The 3D model may define locations and orientations of leaves and stems of plants. The 3D model may be used to determine at least one biometric parameter of at least one plant in the crop. Such detection of plant biometrics may facilitate the automation of crop monitoring and treatment.

First claim

What is claimed is:

1. A system for detecting crop biometrics, the system comprising a crop modeling device communicatively coupled to an imaging device, wherein the crop modeling device comprises processing circuitry configured to:
receive, from the imaging device, a plurality of images of a crop of plants in a field,
generate, based on the plurality of images, a three-dimensional (3D) model of the plants, wherein the 3D model defines locations and orientations of leaves and stems of the plants, and
determine, based on the 3D model, at least one biometric parameter of at least one plant in the crop, wherein the processor is configured to generate the 3D model of the plants by at least:
generating an initial point cloud reconstruction of the plants in the field based on the plurality of images,
simulating respective routes of a plurality of randomly selected points of the initial point cloud by:
identifying a ground plane within a 3D space of the initial point cloud reconstruction,
simulating, for each of a plurality of randomly selected points within the initial point cloud, point-by-point movement from an initial position of the selected point within the initial point cloud reconstruction towards the identified ground plane based on a height of respective points within the initial point cloud, and
determining the respective simulated routes for each point of the plurality of randomly selected points based on the simulated point-by-point movement to generate a plurality of simulated respective routes, and

detecting respective stems of the plants based on the plurality of simulated respective routes, wherein detecting the respective stems of the plants based on the plurality of simulated respective routes includes selecting only a subset of respective simulated routes of the plurality of simulated respective routes corresponding to the respective stems of the plants in the field, wherein the subset of the respective simulated routes includes more than one simulated route, and wherein the subset of simulated respective routes of the plurality of simulated respective routes is selected at least in part based on a frequency of simulated respective routes that are common to each other.

2. The system of claim 1 , wherein the plurality of images includes images taken at different locations and different orientations along the field or adjacent to the field.

3. The system of claim 1 , wherein the plurality of images includes images taken at different locations and different orientations along a substantially circular predetermined path about the field.

4. The system of claim 1 , wherein each point of the initial point cloud reconstruction is represented by three spatial dimensions, and three chromatic dimensions.

5. The system of claim 1 , wherein, to generate the initial point cloud reconstruction, the processing circuitry is configured to, prior to detecting the respective stems:
align respective points used for the initial point cloud reconstruction within the 3D space relative to the ground plain identified in the 3D space; and
skeletonize the respective points aligned within the 3D space to reduce the number of points in the initial point cloud reconstruction.

6. The system of claim 1 , wherein the 3D model comprises a plurality of labels, each respective label of the plurality of labels associated with a respective element of the 3D model, each label defining the respective element of the 3D model as being part of a stems, a leaf, or soil.

7. The system of claim 1 , further comprising an unmanned vehicle comprising the imaging device, wherein the unmanned vehicle comprises an unmanned aerial vehicle or an unmanned ground vehicle.

8. A method for detecting crop biometrics, the method comprising:
receiving, by processing circuitry and from an imaging device, a plurality of images of a crop of plants in a field;
generating, by the processing circuitry, based on the plurality of images, a three-dimensional (3D) model of the plants, wherein the 3D model defines locations and orientations of leaves and stems of the plants; and
determining, by the processing circuitry, based on the 3D model, at least one biometric parameter of at least one plant in the crop,
wherein generating the 3D model of the plants comprises:
generating an initial point cloud reconstruction of the plants in the field based on the plurality of images,
simulating respective routes of a plurality of randomly selected points of the initial point cloud, wherein simulating the respective routes comprises:
identifying a ground plane within a 3D space of the initial point cloud reconstruction,
simulating, for each of a plurality of randomly selected points within the initial point cloud, point-by-point movement from an initial position of the selected point within the initial point cloud reconstruction towards the identified ground plane based on a height of respective points within the initial point cloud, and
determining the respective simulated routes for each point of the plurality of randomly selected points based on the simulated point-by-point movement to generate a plurality of simulated respective routes, and

detecting respective stems of the plants based on the plurality of simulated respective routes, wherein detecting the respective stems of the plants based on the plurality of simulated respective routes includes selecting only a subset of respective simulated routes of the plurality of simulated respective routes corresponding to the respective stems of the plants in the field, wherein the subset of the respective simulated routes includes more than one simulated route, and wherein the subset of simulated respective routes of the plurality of simulated respective routes is selected at least in part based on a frequency of simulated respective routes that are common to each other.

9. The method of claim 8 , wherein the plurality of images includes images taken at different locations and different orientations along the field or adjacent to the field.

10. The method of claim 8 , wherein the plurality of images includes images taken at different locations and different orientations along a substantially circular predetermined path about the field.

11. The method of claim 8 , wherein each point of the point cloud reconstruction is represented by three spatial dimensions, and three chromatic dimensions.

12. The method of claim 8 , further comprising, prior to detecting the respective stems:
aligning respective points used for the initial point cloud reconstruction within the 3D space relative to the ground plain identified in the 3D space; and
skeletonizing, by the processing circuitry, the respective points aligned within the 3D space to reduce the number of points in the point cloud reconstruction.

13. The method of claim 8 , wherein the 3D model comprises a plurality of labels, each respective label of the plurality of labels associated with a respective element of the 3D model, each label defining the respective element of the 3D model as being part of a stems, a leaf, or soil.

14. The method of claim 8 , wherein the imaging device is part of an unmanned vehicle, and wherein the unmanned vehicle comprises an unmanned aerial vehicle or an unmanned ground vehicle.

15. A crop modeling device comprising processing circuitry and a non-transitory computer readable storage medium comprising instructions, that when executed, cause the processing circuitry to:
receive, from an imaging device, a plurality of images of a crop of plants in a field,
generate, based on the plurality of images, a three-dimensional (3D) model of the plants

Assignees

• Univ Minnesota

Inventors

• Papanikolopoulos Nikolaos
• Morellas Vassilios
• Zermas Dimitris
• Mulla David
• Bazakos Mike

Classifications

• G06Q50/02 Primary

  Agriculture; Fishing; Forestry; Mining · CPC title

• G06V10/763

  Non-hierarchical techniques, e.g. based on statistics of modelling distributions · CPC title

• G06V20/17

  taken from planes or by drones · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06V20/188

  Vegetation · CPC title