System and method to determine a teat position using edge detection in rear images of a livestock from two cameras

Patent metadata

Field	Value
Publication number	US-9681634-B2
Application number	US-201213448751-A
Country	US
Kind code	B2
Filing date	Apr 17, 2012
Priority date	Apr 28, 2011
Publication date	Jun 20, 2017
Grant date	Jun 20, 2017

Abstract

A system includes a first camera, a second camera, and a processor wherein the second camera has a higher resolution than the first camera. The processor is communicatively coupled to the first camera and the second camera and is operable to determine a center coordinate of an udder of a dairy livestock based at least in part upon visual data captured by the first camera. The processor is also operable to determine a position of a teat of the dairy livestock based at least in part upon the center coordinate and visual data captured by the second camera.

First claim

What is claimed is:

1. A system, comprising:
a first camera operable to capture an image of a rear of a dairy livestock, the image comprising a plurality of pixels having depth values;
a second camera, the second camera having a higher resolution than the first camera; and
a processor communicatively coupled to the first camera and the second camera, the processor operable to:
determine edges of the rear of the dairy livestock based at least in part upon the depth values of the plurality of pixels;
determine a center coordinate of an udder of the dairy livestock based at least in part upon the determined edges of the rear of the dairy livestock; and
determine a position of a teat of the dairy livestock based at least in part upon the center coordinate and visual data captured by the second camera.

2. The system of claim 1 , wherein the position of the teat is relative to the center coordinate.

3. The system of claim 1 , further comprising a memory communicatively coupled to the processor, the memory operable to store a plurality of coordinates comprising an x-coordinate, a y-coordinate, and a z-coordinate for each teat of the dairy livestock, wherein each coordinate is in relation to the center of the udder of the dairy livestock.

4. The system of claim 1 , wherein the first camera is a three-dimensional camera and the second camera is a two-dimensional camera.

5. The system of claim 1 , further comprising a tracker operable to track the rear of the dairy livestock relative to the rear of a milking stall.

6. The system of claim 5 , further comprising an encoder communicatively coupled to the tracker and operable to determine a distance that the tracker moves.

7. A method, comprising:
capturing an image of a rear of a dairy livestock, the image comprising a plurality of pixels having depth values;
determining edges of the rear of the dairy livestock based at least in part upon the depth values of the plurality of pixels;
determining a center coordinate of an udder of the dairy livestock based at least in part upon the determined edges of the rear of the dairy livestock; and
determining a position of a teat of the dairy livestock based at least in part upon the center coordinate and visual data captured by a second camera, wherein the second camera has a higher resolution than the first camera.

8. The method of claim 7 , wherein the position of the teat is relative to the center coordinate.

9. The method of claim 7 , further comprising storing a plurality of coordinates comprising an x-coordinate, a y-coordinate, and a z-coordinate for each teat of the dairy livestock, wherein each coordinate is in relation to the center of the udder of the dairy livestock.

10. The method of claim 7 , wherein the first camera is a three-dimensional camera and the second camera is a two-dimensional camera.

11. The method of claim 7 , further comprising determining the position of the rear of the dairy livestock relative to the rear of a milking stall.

12. A system, comprising:
a memory operable to store an image of a rear of a dairy livestock, the image comprising a plurality of pixels having depth values; and
a processor communicatively coupled to the memory, the processor operable to:
determine edges of the rear of the dairy livestock based at least in part upon the depth values of the plurality of pixels;
determine a center coordinate of an udder of the dairy livestock based at least in part upon the determined edges of the rear of the dairy livestock;
determine a z-coordinate of a reference point of a teat of the diary livestock based at least in part upon stored coordinates of a dairy livestock and the determined center coordinate of the udder of the dairy livestock;
determine an x-coordinate of the reference point based at least in part upon the stored coordinates and the determined center coordinate of the dairy livestock; and
determine a y-coordinate of the reference point based at least in part upon the stored coordinates and the determined center coordinate of the dairy livestock.

13. The system of claim 12 , further comprising a two-dimensional camera comprising a laser device, wherein the two-dimensional camera is communicatively coupled to the processor.

14. The system of claim 13 , further comprising a robotic arm, wherein the two-dimensional camera is attached to the robotic arm.

15. The system of claim 14 , wherein the processor is further operable to control the robotic arm to move to the reference point.

16. The system of claim 13 , wherein the processor is further operable to:
determine a z-coordinate of the teat based at least in part upon visual data captured by the two-dimensional camera;
determine a y-coordinate of the teat based at least in part upon the visual data captured by the two-dimensional camera; and
determine an x-coordinate of the teat based at least in part upon the y-coordinate of the teat and the visual data captured by the two-dimensional camera.

17. The system of claim 16 , wherein determining the y-coordinate comprises determining the tip of the teat.

18. The system of claim 16 , wherein the visual data captured by the two-dimensional camera comprises a reflection of a laser emitted by the laser device.

19. The system of claim 18 , wherein determining the x-coordinate comprises:
determining an angle based at least in part upon the laser; and
determining a distance of the teat from the camera based at least in part upon the angle and the y-coordinate of the teat.

20. The system of claim 12 , further comprising a tracker operable to track the rear of the dairy livestock relative to the rear of a milking stall.

21. The system of claim 20 , wherein the displacement measurement is based at least in part on the tracker.

22. A method comprising:
storing an image of a rear of a dairy livestock, the image comprising a plurality of pixels having depth values;
determining edges of the rear of the dairy livestock based at least in part upon the depth values of the plurality of pixels;
determining a center coordinate of an udder of the dairy livestock based at least in part upon the determined edges of the rear of the dairy livestock;
determining a z-coordinate of a reference point, using a processor, of a teat of the diary livestock based at least in part upon stored coordinates of a dairy livestock and the determined center coordinate of the udder of the dairy livestock;
determining an x-coordinate of the reference point based at least in part upon the stored coordinates and the determined center coordinate of the dairy livestock; and
determining a y-coordinate of the reference point based at least in part upon the stored coordinates and the determined center coordinate of the dairy livestock.

23. The method of claim 22 , wherein the processor is communicatively coupled to a two-dimensional camera, the two-dimensional camera comprising a laser device.

24. The method of claim 23 , wherein the two-dimensional camera is coupled to a robotic arm.

25. The method of claim 24 , further comprising controlling a robotic arm to move to the reference point.

26. The method of claim 23 , further comprising:
determining a z-coordinate of the teat based at least in part upon visual data captured by the two-dimensional camera;
determining a y-coordinate of the teat based at least in part upon the visual data captured by the two-dimensional camera; and
determini

Assignees

• Hofman Henk
• De Ruijter Cor
• Koekoek Menno
• Van Der Sluis Peter Willem
• Technologies Holdings Corp

Inventors

• Hofman Henk
• De Ruijter Cor
• Koekoek Menno
• Van Der Sluis Peter Willem

Classifications

• G06T7/75

  involving models · CPC title

• G06T2207/30128

  Food products · CPC title

• A01J5/0175 Primary

  Attaching of clusters · CPC title

• A01J5/007

  Monitoring milking processes; Control or regulation of milking machines · CPC title