Position-controlled robotic fleet with visual handshakes

Patent metadata

Field	Value
Publication number	US-9465390-B2
Application number	US-201414538047-A
Country	US
Kind code	B2
Filing date	Nov 11, 2014
Priority date	Nov 11, 2014
Publication date	Oct 11, 2016
Grant date	Oct 11, 2016

Abstract

Example methods and systems may provide for a system that includes a control system communicatively coupled to a first robotic device and a second robotic device. The control system may identify a collaborative operation to be performed by a first robotic device and a second robotic device that is based on a relative positioning between the first robotic device and the second robotic device. The control system may also determine respective locations of the first robotic device and the second robotic device. The control system may further initiate a movement of the first robotic device along a path from the determined location of the first robotic device towards the determined location of the second robotic device. The first robotic device and the second robotic device may then establish a visual handshake that indicates the relative positioning between the first robotic device and the second robotic device for the collaborative operation.

First claim

What is claimed is:

1. A method comprising:
identifying, by a control system, a collaborative operation to be performed by a first robotic device and a second robotic device, wherein the collaborative operation is based on a relative positioning between the first robotic device and the second robotic device;
based on data representing a map of an environment in which the first robotic device and the second robotic device are operating, determining, by the control system, respective locations, relative to the map, of the first robotic device and the second robotic device, wherein the map is updated in response to movements of the first robotic device and the second robotic device relative to the environment;
initiating, by the control system, a movement of the first robotic device along a path from the determined respective location of the first robotic device relative to the map towards the determined respective location relative to the map of the second robotic device; and
after initiating the movement, causing, by the control system, the first robotic device and the second robotic device to establish a visual handshake that indicates the relative positioning between the first robotic device and the second robotic device for the collaborative operation.

2. The method of claim 1 , wherein causing the first robotic device and the second robotic device to establish the visual handshake comprises:
causing the second robotic device to orient a camera along the path from the determined respective location of the first robotic device relative to the map towards the determined respective location of the second robotic device.

3. The method of claim 1 , wherein causing the first robotic device and the second robotic device to establish the visual handshake comprises:
causing the first robotic device to orient a camera towards the determined respective location of the second robotic device.

4. The method of claim 1 , wherein causing the first robotic device and the second robotic device to establish the visual handshake comprises:
causing a camera of the first robotic device to generate data representing images from a viewpoint that is oriented towards the determined respective location of the second robotic device relative to the map;
based on the data representing the images from the viewpoint that is oriented towards the determined respective location of the second robotic device relative to the map, detecting one or more particular images that depict a tag on an exterior surface of the second robotic device; and
based on respective sizes and positions of the tag within the one or more particular images, determining the relative positioning between the first robotic device and the second robotic device.

5. The method of claim 1 , wherein the collaborative operation involves positioning the first robotic device at a particular distance from the second robotic device, the method further comprising:
causing at least one of the first robotic device and the second robotic device to move from the relative positioning indicated by the visual handshake to a relative positioning in which the first robotic device is at the particular distance from the second robotic device.

6. The method of claim 1 , wherein the collaborative operation involves positioning the first robotic device at a particular orientation with respect to the second robotic device, the method further comprising:
causing at least one of the first robotic device and the second robotic device to move from the relative positioning indicated by the visual handshake to a relative positioning in which the first robotic device is at the particular orientation with respect to the second robotic device.

7. The method of claim 1 , wherein the collaborative operation further involves positioning an end-effector of the first robotic device at a particular position relative to an end-effector of the second robotic device, and wherein the visual handshake further indicates a relative positioning between the end-effector of the first robotic device and the end-effector of the second robotic device.

8. The method of claim 1 , wherein the collaborative operation involves navigating the first robotic device along a path that intersects a given proximity of the second robotic device, the method further comprising:
based on the relative positioning between the first robotic device and the second robotic device indicated by the visual handshake, navigating the first robotic device past the second robotic device such that the first robotic device is at least a threshold distance from the second robotic device during the navigation.

9. The method of claim 1 , further comprising:
after establishing the visual handshake that indicates the relative positioning between the first robotic device and the second robotic device, transferring movement control of the first robotic device from the control system to a local control system of first robotic device.

10. A first robotic device comprising:
a communications interface;
one or more processors; and
data storage configured to store instructions, that when executed by the one or more processors, cause the first robotic device to:
receive instructions to perform a collaborative operation that is based on a relative positioning between the first robotic device and a second robotic device;
receive, over the communications interface, data indicating a path towards the second robotic device, wherein the path is determined based on data representing a map of an environment in which the first robotic device and the second robotic device are operating, and wherein the map is updated in response to movements of the first robotic device and the second robotic device relative to the environment;
responsive to receiving the data indicating the path towards the second robotic device, initiate movement of the first robotic device along the path determined based on data representing the map towards the second robotic device; and
after initiating the movement of the first robotic device along the path, establish a visual handshake with the second robotic device that indicates the relative positioning between the first robotic device and the second robotic device for the collaborative operation.

11. The first robotic device of claim 10 , wherein the instructions further cause the first robotic device to:
receive data indicating a direction in which the second robotic device is located relative to the first robotic device; and
orient a camera in the direction in which the second robotic device is located relative to the first robotic device.

12. The first robotic device of claim 10 , wherein the first robotic device further comprises a camera, and wherein the instructions further cause the first robotic device to:
cause the camera to generate data representing images from a viewpoint that is oriented in the direction in which the second robotic device is located relative to the first robotic device;
based on the data representing the images, detect one or more particular images that depict a tag on an exterior surface of the second robotic device; and
based on respective sizes of the tag within the one or more particular images, determine the relative positioning between the first robotic device and the second robotic device.

13. The first robotic device of claim 10 , wherein establishing the visual handshake with the second robotic device comprises:
determining the relative positioning between the first robotic device and the second robotic device; and
sending data indicating the relative positioning between the first robotic device and the second robotic device to the

Assignees

• Google Inc

Inventors

• Mason Julian
• Konolige Kurt

Classifications

• G05B2219/39146

  Swarm, multiagent, distributed multitask fusion, cooperation multi robots · CPC title

• G05B2219/40605

  Two cameras, each on a different end effector to measure relative position · CPC title

• G05B2219/39129

  One manipulator holds one piece, other inserts, screws other piece, dexterity · CPC title

• B65G1/1373

  for fulfilling orders in warehouses · CPC title

• G05B19/41815 Primary

  characterised by the cooperation between machine tools, manipulators and conveyor or other workpiece supply system, workcell · CPC title