Vision system with teat candidate identification

Patent metadata

Field	Value
Publication number	US-9980457-B2
Application number	US-201615239597-A
Country	US
Kind code	B2
Filing date	Aug 17, 2016
Priority date	Aug 17, 2016
Publication date	May 29, 2018
Grant date	May 29, 2018

Abstract

A system that includes a laser, a three-dimensional (3D) camera, a memory, and a processor. The processor is configured to determine the position of a first teat candidate relative to a second teat candidate, assign the first teat candidate and the second teat candidate as a left teat candidate or right teat candidate, and receive a teat identifier for a target teat. The processor is configured to determine whether the left or right teat candidate is within a teat location range of the target teat and to link the teat identifier with the left teat candidate or right teat candidate based on the determination.

First claim

The invention claimed is:

1. A vision system comprising:
a robotic arm;
a laser disposed on the robotic arm;
a three-dimensional (3D) camera disposed on the robotic arm;
a memory configured to store:
teat location information for a plurality of teats of a dairy livestock; and
teat candidate position information for a first teat candidate and a second teat candidate; and

a processor operably coupled to the robotic arm, the laser, the 3D camera, and the memory, and configured to:
send instructions to the robotic arm to position the laser and the 3D camera adjacent to a dairy livestock;
send an electrical signal that initiates scanning at least a portion of the dairy livestock using at least one of the laser and the 3D camera;
generate the teat candidate position information for the first teat candidate and the second teat candidate based on the scan of the at least a portion of the dairy livestock;
determine the position of the first teat candidate relative to the second teat candidate based on the teat candidate position information;
assign the first teat candidate as a left teat candidate and the second teat candidate as a right teat candidate when the first teat candidate is to the left of the second teat candidate;
assign the first teat candidate as the right teat candidate and the second teat candidate as the left teat candidate when the first teat candidate is to the right of the second teat candidate;
receive a teat identifier for a target teat from the plurality of teats of the dairy livestock, wherein the teat identifier indicates whether the target teat is a left teat or a right teat of the dairy livestock;
determine whether the left teat candidate is within a teat location range of the target teat when the target teat is a left teat;
link the teat identifier with the left teat candidate when the left teat candidate is within the teat location range of the target teat and the target teat is a left teat;
link the teat identifier with the right teat candidate when the left teat candidate is outside of the teat location range of the target teat and the target teat is a left teat;
determine whether the right teat candidate is within the teat location range of the target teat when the target teat is a right teat;
link the teat identifier with the right teat candidate when the right teat candidate is within the teat location range of the target teat and the target teat is a right teat;
link the teat identifier with the left teat candidate when the right teat candidate is outside of the teat location range of the target teat and the target teat is a right teat; and
send instructions to the robotic arm to move a portion of the robotic arm toward the target teat based on the teat candidate position information for the teat candidate linked with the teat identifier.

2. The system of claim 1 , wherein the processor is configured to store in the memory:
a relationship between the target teat and the left teat candidate when the teat identifier is linked to the left teat candidate; and
a relationship between the target teat and the right teat candidate when the teat identifier is linked to the right teat candidate.

3. The system of claim 1 , wherein:
the 3D camera is configured to generate a 3D image of the dairy livestock; and
the processor is configured to:
process the 3D image to obtain the teat candidate position information; and
store the teat candidate position information in the memory.

4. The system of claim 1 , wherein:
the laser is configured to generate a profile signal of a portion of the dairy livestock; and
the processor is configured to:
process the profile signal to obtain the teat candidate position information; and
store the teat candidate position information in the memory.

5. The system of claim 1 , wherein the processor is configured to:
discard the left teat candidate when the left teat candidate is outside of the teat location range of the target teat and the target teat is a left teat; and
discard the right teat candidate when the right teat candidate is outside of the teat location range of the target teat and the target teat is a right teat.

6. The system of claim 1 , wherein receiving the teat identifier is based on a determination of which teat from the plurality of teats of the dairy livestock is closest to the first teat candidate and the second teat candidate.

7. The system of claim 1 , wherein the teat identifier indicates the target teat is one of a front right teat of the dairy livestock, a front left teat of the dairy livestock, a rear right teat of the dairy livestock, and a rear left teat of the dairy livestock.

8. An apparatus comprising:
a memory configured to store:
teat location information for a plurality of teats of a dairy livestock; and
teat candidate position information for a first teat candidate and a second teat candidate; and

a processor operably coupled to the memory and a robotic arm, and configured to:
send instructions to the robotic arm to position a laser and a three-dimensional (3D) camera adjacent to a dairy livestock;
send an electrical signal that initiates scanning at least a portion of the dairy livestock using at least one of the laser and the 3D camera;
generate the teat candidate position information for the first teat candidate and the second teat candidate based on the scan of the at least a portion of the dairy livestock;
determine the position of the first teat candidate relative to the second teat candidate based on the teat candidate position information;
assign the first teat candidate as a left teat candidate and the second teat candidate as a right teat candidate when the first teat candidate is to the left of the second teat candidate;
assign the first teat candidate as the right teat candidate and the second teat candidate as the left teat candidate when the first teat candidate is to the right of the second teat candidate;
receive a teat identifier for a target teat from the plurality of teats of the dairy livestock, wherein the teat identifier indicates whether the target teat is a left teat or a right teat of the dairy livestock;
determine whether the left teat candidate is within a teat location range of the target teat when the target teat is a left teat;
link the teat identifier with the left teat candidate when the left teat candidate is within the teat location range of the target teat and the target teat is a left teat;
link the teat identifier with the right teat candidate when the left teat candidate is outside of the teat location range of the target teat and the target teat is a left teat;
determine whether the right teat candidate is within the teat location range of the target teat when the target teat is a right teat;
link the teat identifier with the right teat candidate when the right teat candidate is within the teat location range of the target teat and the target teat is a right teat;
link the teat identifier with the left teat candidate when the right teat candidate is outside of the teat location range of the target teat and the target teat is a right teat; and
send instructions to the robotic arm to move a portion of the robotic arm toward the target teat based on the teat candidate position information for the teat candidate linked with the teat identifier.

9. The apparatus of claim 8 , wherein the processor is configured to store in the memory:
a relationship between the target teat and the left teat candidate when the teat identifier is linked to the left teat candidate; and
a relationship between the target teat and the right teat candidate when the teat identifier is linked to the right teat candidate.

10. The apparatus of claim 8 , wh

Assignees

• Technologies Holdings Corp

Inventors

• Foresman Mark A
• Prevost Bradley J

Classifications

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• A01J5/0175

  Attaching of clusters · CPC title

• A01J5/007 Primary

  Monitoring milking processes; Control or regulation of milking machines · CPC title

• G06T7/75

  involving models · CPC title

• G06T7/11

  Region-based segmentation · CPC title