Tool-pickup system, method, computer program and non-volatile data carrier

Patent metadata

Field	Value
Publication number	US-11991982-B2
Application number	US-201917290704-A
Country	US
Kind code	B2
Filing date	Oct 28, 2019
Priority date	Nov 1, 2018
Publication date	May 28, 2024
Grant date	May 28, 2024

Abstract

Tools in an automatic milking arrangement are picked up by using a robotic arm (110). The robotic arm (110) moves a camera (130) to an origin location (PC) from which the camera (130) registers three-dimensional image data (Dimg3D) of at least one tool (141, 142, 143, 144). The three-dimensional image data is processed using an image-based object identification algorithm to identify objects in the form of the tools and hoses (152). In response to identifying at least one tool, a respective tool position (PT1, PT3, PT4) is determined for each identified tool based on the origin location (PC) and the three-dimensional image data. Then, a grip device (115) is exclusively controlled to the one or more of the respective tool positions (PT1, PT3, PT4) to perform a pick-up operation. Thus, futile attempts to pick-up non-existing or blocked tools can be avoided.

First claim

The invention claimed is:

1. A tool-pickup system for an automatic milking arrangement, the tool-pickup system comprising:
a robotic arm ( 110 ) provided with a grip device ( 115 ) configured to pick up tools ( 141 , 142 , 143 , 144 ), and a camera ( 130 ) configured to register three-dimensional image data (Dimg3D); and
a control unit ( 120 ) operatively connected to the robotic arm, the control unit ( 120 ) configured to:
control the robotic arm ( 110 ) to move the camera ( 130 ) to an origin location (PC) from which at least one tool of the tools ( 141 , 142 , 143 , 144 ) is expected to be visible within a view field (VF) of the camera ( 130 ),
obtain three-dimensional image data (Dimg3D) registered by the camera ( 130 ) at the origin location (PC),
process the three-dimensional image data (Dimg3D) using an image-based object identification algorithm to identify objects in a form of the tools ( 141 , 143 , 144 ) and/or hoses ( 152 ), and
in response to identifying at least one of the tools ( 141 , 143 , 144 ):
i) determine a respective tool position (PT 1 , PT 3 , PT 4 ) for each identified tool ( 141 , 143 , 144 ) based on the origin location (PC) and the three-dimensional image data (Dimg3D), and
ii) exclusively control the grip device ( 115 ) to one or more of the respective tool positions (PT 1 , PT 3 , PT 4 ) to perform a pick-up operation,
wherein the control unit ( 120 ) is further configured to produce an alert (A) in response to identifying at least one hose ( 152 ) at a position where, in a current stage of a procedure executed by the automatic milking arrangement, one of said tools ( 141 , 142 , 143 , 144 ) should be present.

2. The tool-pickup system according to claim 1 , wherein the control unit ( 120 ) is configured to process the three-dimensional image data (Dimg3D) by searching for the tools ( 141 , 142 , 143 , 144 ) in at least one predefined volume (L, R; 441 , 442 ) within the view field (VF).

3. The tool-pickup system according to claim 2 , wherein the at least one predefined volume comprises a respective line (L) or arc for each of the tools ( 141 , 142 , 143 , 144 ) along which respective line (L) expected tool positions (PET 1 , PET 2 , PET 3 , PET 4 ) are defined within a range (R).

4. The tool-pickup system according to claim 3 , wherein, after having controlled the grip device ( 115 ) to perform a pick-up operation at a particular one of said tool positions (PT 1 , PT 3 , PT 4 ), the control unit ( 120 ) is configured to exclude the predefined volume for said particular one tool position from a subsequent search for at least one remaining tool of said tools in the three-dimensional image data (Dimg3D).

5. The tool-pickup system according to claim 2 , wherein the at least one predefined volume comprises a respective area ( 441 ; 442 ) for each of the tools ( 141 , 142 ) within which respective area ( 441 ; 442 ) expected tool positions (PET 1 , PET 2 ) are defined.

6. The tool-pickup system according to claim 5 , wherein, after having controlled the grip device ( 115 ) to perform a pick-up operation at a particular one of said tool positions (PT 1 , PT 3 , PT 4 ), the control unit ( 120 ) is configured to exclude the predefined volume for said particular one tool position from a subsequent search for at least one remaining tool of said tools in the three-dimensional image data (Dimg3D).

7. The tool-pickup system according to claim 2 , wherein, after having controlled the grip device ( 115 ) to perform a pick-up operation at a particular one of said tool positions (PT 1 , PT 3 , PT 4 ), the control unit ( 120 ) is configured to exclude the predefined volume for said particular one tool position from a subsequent search for at least one remaining tool of said tools in the three-dimensional image data (Dimg3D).

8. The tool-pickup system according to claim 1 , wherein the tools comprise at least one of:
one or more teatcups and one or more cleaning cups.

9. The tool-pickup system according to claim 1 , wherein the tools comprise at least one teatcup and at least one cleaning cup.

10. A method for picking up tools in an automatic milking arrangement, the method comprising:
controlling a robotic arm ( 110 ) to move a camera ( 130 ) arranged on the robotic arm ( 110 ) to an origin location (PC) from which at least one tool of the tools ( 141 , 142 , 143 , 144 ) is expected to be visible within a view field (VF) of the camera ( 130 );
using the camera ( 13 ) to register three-dimensional image data (Dimg3D) within the view field (VF) of the camera ( 130 );
obtaining the three-dimensional image data (Dimg3D) registered by the camera ( 130 ) at the origin location (PC);
processing the three-dimensional image data (Dimg3D) using an image-based object identification algorithm to identify objects in a form of tools ( 141 , 143 , 144 ) and/or hoses ( 152 ), and
in response to identifying at least one tool of the tools ( 141 , 143 , 144 ):
i) determining a respective tool position (PT 1 , PT 3 , PT 4 ) for each identified tool ( 141 , 143 , 144 ) based on the origin location (PC) and the three-dimensional image data (Dimg3D), and
ii) exclusively controlling a grip device ( 115 ) on the robotic arm ( 110 ) to one or more of the respective tool positions (PT 1 , PT 3 , PT 4 ) to perform a pick-up operation of each respective identified tool ( 141 , 143 , 144 ); and
producing an alert (A) in response to identifying at least one hose ( 152 ) at a position where, in a current stage of a procedure executed by the automatic milking arrangement, one of said tools ( 141 , 142 , 143 , 144 ) should be present.

11. The method according to claim 10 , comprising:
processing the three-dimensional image data (Dimg3D) by searching for each of the tools ( 141 , 142 , 143 , 144 ) in at least one predefined volume (L, R; 441 , 442 ) within the view field (VF).

12. The method according to claim 11 , wherein the at least one predefined volume is represented by a respective line (L) or arc for each of the tools ( 141 , 142 , 143 , 144 ) along which respective line (L) expected tool positions (PET 1 , PET 2 , PET 3 , PET 4 ) are defined within a range (R).

13. The method according to claim 12 , wherein, after having controlled the grip device ( 115 ) to perform a pick-up operation at a particular one of said tool positions (PT 1 , PT 3 , PT 4 ), the method comprises excluding the predefined volume for said particular one tool position from a subsequent search for at least one remaining tool of said tools in the three-dimensional image data (Dimg3D).

14. The method according to claim 11 , wherein the at least one predefined volume is represented by a respective area ( 441 ; 442 ) for each of the tools ( 141 , 142 ) within which area ( 441 ; 442 ) expected tool positions (PET 1 , PET 2 ) are defined.

15. The method according to claim 14 , wherein, after having controlled the grip device ( 115 ) to perform a pick-up operation at a particular one of said tool positions (PT 1 , PT 3 , PT 4 ), the method comprises excluding the predefined volume for said particular one tool position from a subsequent search for at least one remaining tool of said tools in the three-dimensional image data (Dimg3D).

16. The method according to claim 11 , wherein, after having controlled the grip device ( 115 ) to perform a pick-up operation at a particular one of said tool positions (PT 1 , PT 3 , PT 4 ), the method comprises excluding the predefined volume for said particular one tool position from a subsequent search for at least one remaining tool of said tools in t

Assignees

• Delaval Holding Ab

Inventors

• Eriksson Andreas

Classifications

• A01J5/007 Primary

  Monitoring milking processes; Control or regulation of milking machines · CPC title

• A01J7/02

  for cleaning or sanitising milking machines or devices · CPC title

• B25J9/1697 Primary

  Vision controlled systems · CPC title

• B25J11/0045

  Manipulators used in the food industry · CPC title

• B25J13/08

  by means of sensing devices, e.g. viewing or touching devices · CPC title