Methods and systems for recognizing machine-readable information on three-dimensional objects

Patent metadata

Field	Value
Publication number	US-9707682-B1
Application number	US-201514950828-A
Country	US
Kind code	B1
Filing date	Nov 24, 2015
Priority date	Mar 15, 2013
Publication date	Jul 18, 2017
Grant date	Jul 18, 2017

Abstract

Methods and systems for recognizing machine-readable information on three-dimensional (3D) objects are described. A robotic manipulator may move at least one physical object through a designated area in space. As the at least one physical object is being moved through the designated area, one or more optical sensors may determine a location of a machine-readable code on the at least one physical object and, based on the determined location, scan the machine-readable code so as to determine information associated with the at least one physical object encoded in the machine-readable code. Based on the information associated with the at least one physical object, a computing device may then determine a respective location in a physical environment of the robotic manipulator at which to place the at least one physical object. The robotic manipulator may then be directed to place the at least one physical object at the respective location.

First claim

What is claimed is:

1. A system comprising:
a robotic arm;
a plurality of mirrors coupled to the robotic arm and arranged to provide reflections of respective surfaces of a physical object;
one or more optical sensors coupled to the robotic arm, the one or more optical sensors being configured to detect, on the plurality of mirrors, the reflections of the respective surfaces of the physical object, wherein the plurality of mirrors includes at least four mirrors that are equally spaced out around the robotic arm so as to enable the one or more optical sensors to acquire images of four surfaces of the physical object simultaneously;
at least one processor; and
data storage comprising instructions executable by the at least one processor to cause the system to perform functions comprising:
determining a position in a designated area of a physical environment of the system to which to move the robotic arm such that the plurality of mirrors are arranged to provide reflections of multiple respective surfaces of the physical object in the designated area;
causing the robotic arm to move to the determined position;
while the physical object is in the designated area, causing the one or more optical sensors to acquire image data indicative of the reflections on the plurality of mirrors of the multiple respective surfaces of the physical object; and
based on the image data, determining a scan of a machine-readable code on the physical object.

2. The system of claim 1 , the functions further comprising:
based on the scan, determining data associated with the physical object encoded in the machine-readable code.

3. The system of claim 2 , wherein the data associated with the physical object includes one or more of: dimensions of the physical object, a weight of the physical object, a respective transport structure on which to place the physical object, a respective transport vehicle on which to place the physical object, and an identifier of at least one other physical object in the physical environment at which to place the physical object proximate to.

4. The system of claim 2 , further comprising at least one robotic manipulator, the functions further comprising:
based on the data associated with the physical object encoded in the machine-readable code, determining a respective location in the physical environment at which to cause the at least one robotic manipulator to place the physical object.

5. The system of claim 4 , the functions further comprising:
causing the at least one robotic manipulator to place the physical object at the respective location after a predetermined period of time from the determining of the data associated with the physical object encoded in the machine-readable code.

6. The system of claim 1 , wherein the image data includes an image corresponding to at least a portion of the machine-readable code based on a reflection of the machine-readable code in the plurality of mirrors, and wherein determining the scan of the machine-readable code comprises:
modifying a scale and orientation of the image,
making a comparison between the modified image and a predetermined machine-readable code template,
based on the comparison, determining a remaining portion of the machine-readable code, and
responsive to determining the remaining portion of the machine-readable code, and based on the image corresponding to at least the portion of the machine-readable code, determining the scan of the machine-readable code.

7. The system of claim 1 , further comprising:
a platform located in the designated area and configured to support an additional physical object, wherein the additional physical object includes an additional machine-readable code located on a bottom surface of the additional physical object; and
an additional optical sensor located underneath the platform and configured to scan the bottom surface of the additional physical object as the additional physical object is supported by the platform,
wherein the functions further comprise:
while the additional physical object is supported by the platform in the designated area, causing the additional optical sensor to acquire additional image data indicative of the bottom surface of the additional physical object; and
based on the additional image data, determining a scan of the additional machine-readable code.

8. The system of claim 1 , wherein at least one optical sensor of the one or more optical sensors is coupled to an end of the robotic arm, the system further comprising:
a plurality of connecting members having respective first ends disposed on a periphery of the end of the robotic arm,
wherein respective mirrors of the plurality of mirrors are connected to respective second ends of the connecting members, the respective second ends being opposite the respective first ends.

9. The system of claim 1 , wherein the data storage includes orientation information associated with the physical object, wherein the orientation information includes an indication of at least one face of the physical object that includes the machine-readable code, and
wherein determining the position in the designated area of the physical environment to which to move the robotic arm such that the plurality of mirrors coupled to the robotic arm are arranged to provide the reflections of the multiple respective surfaces of the physical object in the designated area comprises based on the orientation information, determining the position to be a position in the designated area of the physical environment to which to move the robotic arm such that the plurality of mirrors coupled to the robotic arm are arranged to provide the reflections of the at least one face of the physical object.

10. A non-transitory computer readable medium having stored thereon instructions that, upon execution by a computing system, cause the computing system to perform functions comprising:
receiving, from data storage accessible by the computing system, orientation information associated with a physical object, wherein the orientation information includes an indication of at least one face of the physical object that includes a machine-readable information;
based on the orientation information, determining a position in a designated area of a physical environment to which to move a robotic arm such that a plurality of mirrors coupled to the robotic arm are arranged to provide reflections of the at least one face of the physical object in the designated area, wherein one or more optical sensors are coupled to the robotic arm, the one or more optical sensors being configured to detect, on the plurality of mirrors, the reflections of the at least one face of the physical object;
causing the robotic arm to move to the determined position;
while the physical object is in the designated area, causing the one or more optical sensors to acquire image data indicative of the reflections on the plurality of mirrors of the at least one face of the physical object; and
based on the image data, determining a scan of the machine-readable information included on the physical object.

11. The non-transitory computer readable medium of claim 10 , wherein the machine-readable information included on the physical object includes at least one text string, the functions further comprising:
based on the image data, determining the at least one text string using optical character recognition (OCR), and
determining data associated with the physical object based on the at least one text string.

12. The non-transitory computer readable medium of claim 10 , the functions further comprising:
based on the scan, determining data associated with the physical o

Assignees

• Google Inc
• X Dev Llc

Inventors

• Konolige Kurt
• Rublee Ethan
• Bradski Gary

Classifications

• B25J9/1671 Primary

  characterised by simulation, either to verify existing program or to create and verify new program, CAD/CAM oriented, graphic oriented programming systems · CPC title

• B25J9/163

  learning, adaptive, model based, rule based expert control · CPC title

• G06T7/55

  from multiple images · CPC title

• B65G47/46

  and distributing, e.g. automatically, to desired points (in tube mail systems B65G51/36) · CPC title

• B25J19/00

  Accessories fitted to manipulators, e.g. for monitoring, for viewing; Safety devices combined with or specially adapted for use in connection with manipulators (safety-devices in general F16P; protection against radiation in general G21F) · CPC title