Selector for robot-retrievable items

Patent metadata

Field	Value
Publication number	US-12285877-B2
Application number	US-202118041012-A
Country	US
Kind code	B2
Filing date	Aug 9, 2021
Priority date	Aug 11, 2020
Publication date	Apr 29, 2025
Grant date	Apr 29, 2025

Abstract

A retrieval controller is disclosed for identifying an item to be retrieved from a flat storage surface by a robot for identifying one item, amongst items stored on a common surface that can be retrieved by a robot equipped with a lateral-motion gripper. The retrieval controller includes: a depth map computing unit configured to establish a global coordinate system, and establish an orthonormal set of basis vectors u, v and w defined in the global coordinate system, where w is approximately orthogonal to the surface that the items are stored on; a vector determination unit; an item selection unit; and a robot instructing unit configured to instruct the robot to retrieve the item based on uv coordinates of one or more w-aligned edges having associated quadrilateral-based, right prisms which do not have interiors that intersect any right, enclosing prisms selected by the item selection unit.

First claim

The invention claimed is:

1. A retrieval controller for identifying an item to be retrieved from a flat storage surface by a robot, the retrieval controller comprising:
a depth map computing unit configured to: establish a global coordinate system, establish an orthonormal set of basis vectors u, v and w defined in the global coordinate system where w is selected to be approximately orthogonal to a surface on which items will be stored, receive a depth map from a depth sensor, convert the received depth map into a 3D Point Cloud defined in the global coordinate system, compute a representation of a partitioning into segments of 3D Points of a 3D Point Cloud such that a segment contains a pair of 3D Points only if the 3D points are to be considered to be part of the surface of a same item;
a prism calculating unit configured to compute a right, enclosing prism for each segment;
a vector determination unit configured to compute each of:
a. an outwards-pointing normal of each w-aligned face of each computed right enclosing prism, and
b. an outwards-pointing normal of each w-aligned edge of each computed right, enclosing prism,
c. wherein w-aligned edges of each computed right, enclosing prism correspond to grasp points that will be precluded from an item selection process;

an item selection unit configured to: iterate over the w-aligned edges of each right enclosing prism that do not correspond to grasp points that are precluded from the item selection process, compute a pair of quadrilateral-based right prisms for each such w-aligned edge, and check whether or not an interior of either of the two quadrilateral-based right prisms associated with a w-aligned edge intersects any of the right enclosing prisms; and
a robot instructing unit configured to instruct the robot to retrieve an item based on uv coordinates of one or more w-aligned edges whose associated quadrilateral-based right prisms do not have interiors that intersect any of the right enclosing prisms selected by the item selection unit.

2. The retrieval controller according to claim 1 , wherein a base of each right enclosing prism is a simple polygon that is orthogonal to w.

3. The retrieval controller according to claim 1 , wherein each right enclosing prism contains all of the 3D Points that constitute its associated segment.

4. The retrieval controller according to claim 1 , wherein each right enclosing prism has minimal volume.

5. The retrieval controller according to claim 1 , wherein the pair of quadrilateral-based right prisms associated with any given w-aligned edge are adjacent to that edge's two neighbouring w-aligned faces.

6. The retrieval controller according to claim 1 , wherein bases of each quadrilateral-based right prism associated with any given w-aligned edge are parallel to a uv plane.

7. The retrieval controller according to claim 1 , wherein each quadrilateral-based right prism associated with any given w-aligned edge has four edges that are parallel to that w-aligned edge and wherein one of these four edges lies in a plane containing that w-aligned edge and that w-aligned edge's normal.

8. The retrieval controller according to claim 1 , wherein for each quadrilateral-based, right prism associated with any given w-aligned edge, associated minimal and maximal projections of quadrilateral-based right prism vertices onto the w axis are equal to minimal and maximal projections of the w-aligned edge onto the w axis.

9. The retrieval controller according to claim 1 , wherein the vector determination unit is configured to identify which w-aligned edges of a computed right enclosing prism correspond to grasp points that should be precluded from an item selection process, the vector determination unit being configured to:
a. compute uv coordinates of a first centroid of a flat storage surface;
b. compute a second centroid of the uv coordinates of the right enclosing prism's vertices; and
c. identify the one or more w-aligned edges whose normals are closest in orientation to a vector that runs from the first centroid to the second centroid.

10. The retrieval controller according to claim 1 , wherein the vector determination unit is configured to identify which w-aligned edges of a computed right enclosing prism correspond to grasp points that should be precluded from the item selection process, the vector determination unit being configured to:
a. compute a minimal area convex polygon K that encloses a storage surface's projection onto the uv plane;
b. compute uv coordinates of vertices of the right, enclosing prism;
c. exclude a given w-aligned edge's grasp points from the item selection process only if one of the following conditions holds:
i. the minimal area convex polygon K encloses a projected edge of the right enclosing prism, and translating that projected edge a predetermined distance in an opposite direction of the w-aligned edge's normal would cause the projected edge to cross an edge of the minimal area convex polygon K; or
ii. part of a projected edge of the right, enclosing prism lies outside the minimal area convex polygon K, and translating that projected edge a user-specified distance in the opposite direction of the w-aligned edge's normal would move the projected edge further from a closest edge of the minimal area convex polygon K.

11. A grasping system in combination with the retrieval controller of claim 1 , the system comprising:
a depth sensor for generating a depth map of an item stored on one or more stacked surfaces;
a robot for grasping the item; and
the retrieval controller, configured to control the robot to grasp the item.

12. A storage system in combination with the system of claim 11 , the storage system comprising:
a first set of parallel rails or tracks extending in an X-direction, and a second set of parallel rails or tracks extending in a Y-direction transverse to the first set in a substantially horizontal plane to form a grid pattern including a plurality of grid spaces;
a plurality of stacks of containers located beneath the rails, and arranged such that each stack is located within a footprint of a single grid space;
at least one transporting device, the at least one transporting device being configured to selectively move in the X and/or Y directions, above the stacks on the rails and configured to transport a container;
a picking station configured to receive a container transported by the at least one transporting device; and
the grasping system, wherein the grasping system is arranged to grasp an item and place it in a container at the picking station.

13. The storage system according to claim 12 , wherein the at least one transporting device has a footprint that occupies only a single grid space in the storage system, such that a transporting device occupying one grid space does not obstruct a transporting device occupying or traversing adjacent grid spaces in the X and/or Y directions.

14. A method of identifying an item to be retrieved from a flat storage surface by a robot, the method comprising:
a. establishing a global coordinate system;
b. establishing an orthonormal set of basis vectors u, v and w defined in the global coordinate system, where w is approximately orthogonal to the surface that items are stored on;
c. receiving a depth map from a depth sensor;
d. converting the received depth map into a 3D Point Cloud defined in the global coordinate system;
e. computing a representation of a partitioning into segments of 3D Points of the 3D Point Cloud that were observed to be lying on surfaces of items that are to be retrieved, such that a segment contains a pair of 3D Points only if

Assignees

• Ocado Innovation Ltd

Inventors

• Pedro Osemwaro Jeremiah Oghenetega

Classifications

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• B25J9/1612 Primary

  characterised by the hand, wrist, grip control · CPC title

• G06T7/13

  Edge detection · CPC title

• G06T7/50

  Depth or shape recovery · CPC title

• Y02P90/60

  Electric or hybrid propulsion means for production processes · CPC title