Tool-pickup system, method, computer program and non-volatile data carrier

Patent metadata

Field	Value
Publication number	US-2022015326-A1
Application number	US-201917290704-A
Country	US
Kind code	A1
Filing date	Oct 28, 2019
Priority date	Nov 1, 2018
Publication date	Jan 20, 2022
Grant date	—

Abstract

Tools in an automatic milking arrangement are picked up by using a robotic arm (110). The robotic arm (110) moves a camera (130) to an origin location (PC) from which the camera (130) registers three-dimensional image data (Dimg3D) of at least one tool (141, 142, 143, 144). The three-dimensional image data is processed using an image-based object identification algorithm to identify objects in the form of the tools and hoses (152). In response to identifying at least one tool, a respective tool position (PT1, PT3, PT4) is determined for each identified tool based on the origin location (PC) and the three-dimensional image data. Then, a grip device (115) is exclusively controlled to the one or more of the respective tool positions (PT1, PT3, PT4) to perform a pick-up operation. Thus, futile attempts to pick-up non-existing or blocked tools can be avoided.

First claim

1 . A tool-pickup system for an automatic milking arrangement, the tool-pickup system comprising:
a robotic arm ( 110 ) provided with a grip device ( 115 ) configured to pick up tools ( 141 , 142 , 143 , 144 ), and a camera ( 130 ) configured to register three-dimensional image data (Dimg 3 D); and a control unit ( 120 ) operatively connected to the robotic arm, the control unit ( 120 ) configured to: control the robotic arm ( 110 ) to move the camera ( 130 ) to an origin location (PC) from which at least one tool of the tools ( 141 , 142 , 143 , 144 ) is expected to be visible within a view field (VF) of the camera ( 130 ), obtain three-dimensional image data (Dimg 3 D) registered by the camera ( 130 ) at the origin location (PC), process the three-dimensional image data (Dimg 3 D) using an image-based object identification algorithm to identify objects in a form of the tools ( 141 , 143 , 144 ) and/or hoses ( 152 ), and in response to identifying at least one of the tools ( 141 , 143 , 144 ): i) determine a respective tool position (PT 1 , PT 3 , PT 4 ) for each identified tool ( 141 , 143 , 144 ) based on the origin location (PC) and the three-dimensional image data (Dimg 3 D), and ii) exclusively control the grip device ( 115 ) to one or more of the respective tool positions (PT 1 , PT 3 , PT 4 ) to perform a pick-up operation.

2 . The tool-pickup system according to claim 1 , wherein the control unit ( 120 ) is further configured to produce an alert (A) in response to identifying at least one hose ( 152 ) at a position where, in a current stage of a procedure executed by the automatic milking arrangement, one of said tools ( 141 , 142 , 143 , 144 ) should be present.

3 . The tool-pickup system according to claim 1 , wherein the control unit ( 120 ) is configured to process the three-dimensional image data (Dimg 3 D) by searching for the tools ( 141 , 142 , 143 , 144 ) in at least one predefined volume (L, R; 441 , 442 ) within the view field (VF).

4 . The tool-pickup system according to claim 3 , wherein the at least one predefined volume comprises a respective line (L) or arc for each of the tools ( 141 , 142 , 143 , 144 ) along which respective line (L) expected tool positions (PET 1 , PET 2 , PET 3 , PET 4 ) are defined within a range (R).

5 . The tool-pickup system according to claim 3 , wherein the at least one predefined volume comprises a respective area ( 441 ; 442 ) for each of the tools ( 141 , 142 ) within which respective area ( 441 ; 442 ) expected tool positions (PET 1 , PET 2 ) are defined.

6 . The tool-pickup system according to claim 3 , wherein, after having controlled the grip device ( 115 ) to perform a pick-up operation at a particular one of said tool positions (PT 1 , PT 3 , PT 4 ), the control unit ( 120 ) is configured to exclude the predefined volume for said particular one tool position from a subsequent search for at least one remaining tool of said tools in the three-dimensional image data (Dimg 3 D).

7 . The tool-pickup system according to claim 1 , wherein the tools comprise at least one of:
one or more teatcups and one or more cleaning cups.

8 . A method for picking up tools in an automatic milking arrangement, the method comprising:
controlling a robotic arm ( 110 ) to move a camera ( 130 ) arranged on the robotic arm ( 110 ) to an origin location (PC) from which at least one tool of the tools ( 141 , 142 , 143 , 144 ) is expected to be visible within a view field (VF) of the camera ( 130 ); using the camera ( 13 ) to register three-dimensional image data (Dimg 3 D) within the view field (VF) of the camera ( 130 ); obtaining the three-dimensional image data (Dimg 3 D) registered by the camera ( 130 ) at the origin location (PC); processing the three-dimensional image data (Dimg 3 D) using an image-based object identification algorithm to identify objects in a form of tools ( 141 , 143 , 144 ) and/or hoses ( 152 ), and in response to identifying at least one tool of the tools ( 141 , 143 , 144 ): i) determining a respective tool position (PT 1 , PT 3 , PT 4 ) for each identified tool ( 141 , 143 , 144 ) based on the origin location (PC) and the three-dimensional image data (Dimg 3 D), and ii) exclusively controlling a grip device ( 115 ) on the robotic arm ( 110 ) to one or more of the respective tool positions (PT 1 , PT 3 , PT 4 ) to perform a pick-up operation of each respective identified tool ( 141 , 143 , 144 ).

9 . The method according to claim 8 , comprising:
produce an alert (A) in response to identifying at least one hose ( 152 ) at a position where, in a current stage of a procedure executed by the automatic milking arrangement, one of said tools ( 141 , 142 , 143 , 144 ) should be present.

10 . The method according to claim 8 , comprising:
processing the three-dimensional image data (Dimg 3 D) by searching for each of the tools ( 141 , 142 , 143 , 144 ) in at least one predefined volume (L, R; 441 , 442 ) within the view field (VF).

11 . The method according to claim 10 , wherein the at least one predefined volume is represented by a respective line (L) or arc for each of the tools ( 141 , 142 , 143 , 144 ) along which respective line (L) expected tool positions (PET 1 , PET 2 , PET 3 , PET 4 ) are defined within a range (R).

12 . The method according to claim 10 , wherein the at least one predefined volume is represented by a respective area ( 441 ; 442 ) for each of the tools ( 141 , 142 ) within which area ( 441 ; 442 ) expected tool positions (PET 1 , PET 2 ) are defined.

13 . The method according to claim 10 , wherein, after having controlled the grip device ( 115 ) to perform a pick-up operation at a particular one of said tool positions (PT 1 , PT 3 , PT 4 ), the method comprises excluding the predefined volume for said particular one tool position from a subsequent search for at least one remaining tool of said tools in the three-dimensional image data (Dimg 3 D).

14 . The method according to claim 8 , wherein the tools comprise at least one of:
one or more teatcups and one or more cleaning cups.

15 . A non-transitory data carrier ( 126 ) containing a computer program ( 127 ) loadable into a processing unit ( 125 ), the computer program ( 127 ) comprising software, when executed by the processing unit ( 125 ), causes the processing unit to perform the method according claim 8 .

16 . (canceled)

17 . The tool-pickup system according to claim 4 , wherein, after having controlled the grip device ( 115 ) to perform a pick-up operation at a particular one of said tool positions (PT 1 , PT 3 , PT 4 ), the control unit ( 120 ) is configured to exclude the predefined volume for said particular one tool position from a subsequent search for at least one remaining tool of said tools in the three-dimensional image data (Dimg 3 D).

18 . The tool-pickup system according to claim 5 , wherein, after having controlled the grip device ( 115 ) to perform a pick-up operation at a particular one of said tool positions (PT 1 , PT 3 , PT 4 ), the control unit ( 120 ) is configured to exclude the predefined volume for said particular one tool position from a subsequent search for at least one remaining tool of said tools in the three-dimensional image data (Dimg 3 D).

19 . The tool-pickup system according to claim 1 , wherein the tools comprise at least one teatcup and at least one cleaning cup.

20 . The method according to claim 11 , wherein, after having controlled the grip device ( 115 ) to pe

Assignees

• Delaval Holding Ab

Inventors

• Eriksson Andreas

Classifications

• H04N23/695

  Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects · CPC title

• B25J9/1697 Primary

  Vision controlled systems · CPC title

• G06T1/0014

  Image feed-back for automatic industrial control, e.g. robot with camera (robots B25J19/023) · CPC title

• G08B21/18

  Status alarms (G08B21/02 takes precedence) · CPC title

• G06T2207/30242

  Counting objects in image · CPC title