Vision system with teat detection

Patent metadata

Field	Value
Publication number	US-2019180092-A1
Application number	US-201815947583-A
Country	US
Kind code	A1
Filing date	Apr 6, 2018
Priority date	Aug 17, 2016
Publication date	Jun 13, 2019
Grant date	—

Abstract

A vision system that includes a robotic arm and a three-dimensional (3D) camera operably coupled to a processor. The processor is configured to position the robotic arm adjacent to the dairy livestock and acquire a 3D image using the 3D camera. The processor is further configured to identify a set of teat candidates within the 3D image and to filter the set of teat candidates based on one or more filtering rules. The processor is further configured to determine an aggregate teat candidate score for the consolidated set of teat candidates, compare the aggregate teat candidate score to a score threshold value, and update teat location information for the dairy livestock in response to determining the aggregate teat candidate score is greater than or equal to the score threshold value.

First claim

1 . A vision system comprising:
a robotic arm; a three-dimensional (3D) camera disposed on the robotic arm; a memory operable to store teat location information for a dairy livestock; and a processor operably coupled to the robotic arm, the 3D camera, and the memory, and configured to:
position the robotic arm adjacent to the dairy livestock;
acquire a 3D image using the 3D camera, wherein each pixel of the 3D image is associated with a depth value;
identify a set of teat candidates within the 3D image, wherein each teat candidate is associated with a teat location and a set of pixels in the 3D image;
filter the set of teat candidates, wherein filtering the set of teat candidates comprises:
identifying a set of A-points for a teat candidate, wherein A-points are points within a first distance threshold of the teat location for the teat candidate;
identifying a set of B-points for the teat candidate, wherein B-points are points within a second distance threshold of the teat location for the teat candidate, wherein the second distance threshold is greater than the first distance threshold;
determining whether the teat candidate satisfies one or more filtering rules based on the A-points and the B-points; and
removing the teat candidate from the set of teat candidates in response to determining that the teat candidate fails to satisfy at least one of the filtering rules;

determine an aggregate teat candidate score for the consolidated set of teat candidates, wherein the aggregate teat candidate score is based at least in part on distances between teat candidates and the teat locations for the teats of the dairy livestock;
compare the aggregate teat candidate score to a score threshold value; and
update the teat location information for the dairy livestock in response to determining that the aggregate teat candidate score is greater than or equal to the score threshold value.

2 . The system of claim 1 , wherein determining whether the teat candidate satisfies one or more filtering rules comprises:
determining a total number of A-points and B-points for the teat candidate; comparing the total number of A-points and B-points to a total point count value; and determining the teat candidate satisfies a filtering rule in response to determining that the total number of A-points and B-points is greater than or equal to the total point count value.

3 . The system of claim 1 , wherein determining whether the teat candidate satisfies one or more filtering rules comprises:
determining a total number of A-points for the teat candidate; comparing the total number of A-points an A-point count threshold value; and determining the teat candidate satisfies a filtering rule in response to determining that the total number of A-points is greater than or equal to the A-point count threshold value.

4 . The system of claim 1 , wherein determining whether the teat candidate satisfies one or more filtering rules comprises:
determining a width of the teat candidate based at least in part on the A-points; comparing the width of the teat candidate to a width range limit; and determining the teat candidate satisfies a filtering rule in response to determining that the width of the teat candidate is within the width range limit.

5 . The system of claim 1 , wherein determining whether the teat candidate satisfies one or more filtering rules comprises:
determining a height of the teat candidate based at least in part on the A-points; comparing the height of the teat candidate to a height range limit; and determining the teat candidate satisfies a filtering rule in response to determining that the height of the teat candidate is within the height range limit.

6 . The system of claim 1 , wherein determining whether the teat candidate satisfies one or more filtering rules comprises:
determining a percentage of B-points that are located below the A-points in the 3D image; comparing the determined percentage to a B-point ratio threshold value; and determining the teat candidate satisfies a filtering rule in response to determining that the determined percentage less than or equal to the B-point ratio threshold value.

7 . The system of claim 1 , further comprising consolidating the filtered set of teat candidates, wherein consolidating the filtered set of teat candidates comprises:
identifying a subset of teat candidates having teat locations within a first proximity threshold; and combining the set of A-points and the set of B-points for the teat candidates in the subset of teat candidates.

8 . The system of claim 7 , wherein combining the set of A-points and the set of B-points for the teat candidates in the subset of teat candidates comprises averaging the set of A-points and the set of B-points for the teat candidates in the subset of teat candidates.

9 . A teat identification method, comprising:
positioning, by a processor, a robotic arm adjacent to the dairy livestock, wherein the robotic arm comprises a three-dimensional (3D) camera disposed on the robotic arm; acquiring, by the 3D camera, a 3D image, wherein each pixel of the 3D image is associated with a depth value; identifying, by the processor, a set of teat candidates within the 3D image, wherein each teat candidate is associated with a teat location and a set of pixels in the 3D image; filtering, by the processor, the set of teat candidates, wherein filtering the set of teat candidates comprises:
identifying a set of A-points for a teat candidate, wherein A-points are points within a first distance threshold of the teat location for the teat candidate;
identifying a set of B-points for the teat candidate, wherein B-points are points within a second distance threshold of the teat location for the teat candidate, wherein the second distance threshold is greater than the first distance threshold;
determining whether the teat candidate satisfies one or more filtering rules based on the A-points and the B-points; and
removing the teat candidate from the set of teat candidates in response to determining that the teat candidate fails to satisfy at least one of the filtering rules;
determining, by the processor, an aggregate teat candidate score for the consolidated set of teat candidates, wherein the aggregate teat candidate score is based at least in part on distances between teat candidates and the teat locations for the teats of the dairy livestock; comparing, by the processor, the aggregate teat candidate score to a score threshold value; and updating, by the processor, the teat location information for the dairy livestock in response to determining that the aggregate teat candidate score is greater than or equal to the score threshold value.

10 . The method of claim 9 , wherein determining whether the teat candidate satisfies one or more filtering rules comprises:
determining a total number of A-points and B-points for the teat candidate; comparing the total number of A-points and B-points to a total point count value; and determining the teat candidate satisfies a filtering rule in response to determining that the total number of A-points and B-points is greater than or equal to the total point count value.

11 . The method of claim 9 , wherein determining whether the teat candidate satisfies one or more filtering rules comprises:
determining a total number of A-points for the teat candidate; comparing the total number of A-points an A-point count threshold value; and determining the teat candidate satisfies a filtering rule in response to determining that the total number of A-points is greater than or equal to the A-point count threshold value.

12 . The method of cla

Assignees

• Technologies Holdings Corp

Inventors

• Foresman Mark A
• Voigt Richard B

Classifications

• A01J5/0175 Primary

  Attaching of clusters · CPC title

• G06V40/10

  Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands · CPC title

• G06F18/231

  Hierarchical techniques, i.e. dividing or merging pattern sets so as to obtain a dendrogram · CPC title

• G06F18/23213

  with fixed number of clusters, e.g. K-means clustering · CPC title

• G06F18/2113

  by ranking or filtering the set of features, e.g. using a measure of variance or of feature cross-correlation · CPC title