Methods And Systems For Providing Landmarks To Facilitate Robot Localization And Visual Odometry

Patent metadata

Field	Value
Publication number	US-2016339587-A1
Application number	US-201615226353-A
Country	US
Kind code	A1
Filing date	Aug 2, 2016
Priority date	Aug 25, 2014
Publication date	Nov 24, 2016
Grant date	—

Abstract

Methods and systems for providing landmarks to facilitate robot localization and visual odometry are provided herein. At least one area of a physical environment in which a robotic device resides may be determined to include surfaces that lack sufficient discernable features to determine a location of the at least one area. Instructions may responsively be provided to the robotic device for the robotic device to provide a material in respective patterns onto one or more surfaces of the at least one area. Instructions can responsively be provided for the robotic device to provide the material in respective textures as well. The respective patterns or textures may include sufficient discernable features to determine a location of the at least one area, and the material may remain on the one or more surfaces for a predetermined period of time.

First claim

What is claimed is:

1 . A system comprising:
a robotic device; an applicator configured to provide a material onto a surface of an object, wherein the material, once provided on the surface, is expected to remain on the surface for at least a predetermined period of time; one or more optical sensors; at least one processor; data storage comprising instructions executable by the at least one processor to cause the system to perform operations comprising:
causing the robotic device to provide, using the applicator, the material in a pattern onto one or more surfaces in a physical environment; and
within the predetermined period of time:
detecting, using the one or more optical sensors, the pattern on the one or more surfaces, wherein the pattern includes a discernable feature sufficient to determine a first location of the one or more surfaces in the physical environment;
determining the discernable feature from the detected pattern on the one or more surfaces, and
using the discernable feature to determine the first location of the one or more surfaces with respect to a second location of the robotic device in the physical environment.

2 . The system of claim 1 , wherein the one or more surfaces are one or more surfaces of at least one object, the operations further comprising:
in response to using the discernable feature to determine the first location of the one or more surfaces with respect to the second location of the robotic device in the physical environment, causing the robotic device to manipulate the at least one object.

3 . The system of claim 1 , wherein the one or more surfaces are one or more surfaces of at least one moving object, and
wherein using the discernable feature to determine the first location of the one or more surfaces with respect to the second location of the robotic device in the physical environment comprises using the discernable feature to sense the first location of the at least one moving object with respect to the second location of the robotic device over a period of time as the at least one moving object moves in the physical environment.

4 . The system of claim 1 , further comprising a projector, wherein detecting, using the one or more optical sensors, the pattern on the one or more surfaces comprises:
using the projector to project light onto the one or more surfaces at substantially a location of the pattern, wherein the projected light includes one or more of ultraviolet (UV) light and infrared (IR) light; and while projecting the light onto the one or more surfaces, detecting, using the one or more optical sensors, the pattern on the one or more surfaces.

5 . The system of claim 1 , wherein the pattern is unique.

6 . The system of claim 1 , wherein the material includes one or more of: non-reflective paint, reflective paint, invisible ink, and disappearing ink.

7 . The system of claim 1 , wherein causing the robotic device to provide, using the applicator, the material in the pattern onto the one or more surfaces in the physical environment comprises causing the robotic device to provide, using the applicator, the material as one or more of a two-dimensional (2D) pattern or a three-dimensional (3D) texture onto the one or more surfaces in the physical environment.

8 . The system of claim 1 , wherein causing the robotic device to provide, using the applicator, the material in the pattern onto the one or more surfaces in the physical environment comprises causing the robotic device to first provide a primer material onto the one or more surfaces and subsequently provide, using the applicator, the material in the pattern onto the one or more surfaces.

9 . A method comprising:
causing a robotic device to provide, using an applicator, a material in a pattern onto one or more surfaces in a physical environment, wherein the material, once provided on the one or more surfaces, is expected to remain on the one or more surfaces for at least a predetermined period of time; and within the predetermined period of time:
detecting, using one or more optical sensors, the pattern on the one or more surfaces, wherein the pattern includes a discernable feature sufficient to determine a first location of the one or more surfaces in the physical environment;
determining the discernable feature from the detected pattern on the one or more surfaces, and
using the discernable feature to determine the first location of the one or more surfaces with respect to a second location of the robotic device in the physical environment.

10 . The method of claim 9 , wherein determining the first location of the one or more surfaces with respect to the second location of the robotic device in the physical environment comprises using a simultaneous localization and mapping (SLAM) process to determine (i) the first location of the one or more surfaces with respect to the second location of the robotic device in the physical environment and (ii) a map of at least a portion of the physical environment.

11 . The method of claim 10 , wherein the map includes a visual representation of at least the portion of the physical environment, and
wherein the map includes one or more of: a topological map of the physical environment and a metric map of the physical environment.

12 . The method of claim 9 , further comprising:
before causing the robotic device to provide, using the applicator, the material in the pattern onto the one or more surfaces in the physical environment:
engaging in an attempt to extract at least one discernable feature from at least one image of the one or more surfaces, whereby a number of discernable features are extracted;
based on a determination that the number of discernable features that are extracted is below a threshold, determining that the one or more surfaces lack sufficient discernable features to determine the first location of the one or more surfaces in the physical environment,
wherein causing the robotic device to provide, using the applicator, the material in the pattern onto the one or more surfaces in the physical environment comprises causing the robotic device to provide, using the applicator, the material in the pattern onto the one or more surfaces in the physical environment in response to determining that the one or more surfaces lack sufficient discernable features to determine the first location of the one or more surfaces in the physical environment.

13 . The method of claim 9 , further comprising:
before causing the robotic device to provide, using the applicator, the material in the pattern onto the one or more surfaces in the physical environment:
determining a classification for the one or more surfaces, wherein the classification is representative of (i) a type of surface of the one or more surfaces and (ii) a type of texture of the one or more surfaces; and
based on the classification, determining at least one type of pattern to be provided by the robotic device onto the one or more surfaces,
wherein causing the robotic device to provide, using the applicator, the material in the pattern onto the one or more surfaces in the physical environment comprises causing the robotic device to provide, using the applicator, the material in the pattern of the at least one type of pattern onto the one or more surfaces in the physical environment.

14 . The method of claim 9 , further comprising:
before causing the robotic device to provide, using the applicator, the material in the pattern onto the one or more surfaces in the physical environment:
estimating an amount of time to determine the first location of the one or more surface

Assignees

• Google Inc

Inventors

• Rublee Ethan

Classifications

• B05B13/0431

  with spray heads moved by robots or articulated arms, e.g. for applying liquid or other fluent material to three-dimensional [3D] surfaces · CPC title

• B65G59/02

  De-stacking from the top of the stack · CPC title

• B25J9/1697 Primary

  Vision controlled systems · CPC title

• Y10S901/47

  Optical · CPC title

• B25J19/021

  Optical sensing devices · CPC title