Remote camera-assisted robot guidance

Patent metadata

Field	Value
Publication number	US-11610320-B2
Application number	US-202117173550-A
Country	US
Kind code	B2
Filing date	Feb 11, 2021
Priority date	Feb 19, 2020
Publication date	Mar 21, 2023
Grant date	Mar 21, 2023

Abstract

Methods, systems, and apparatus for remote camera-assisted robot guidance are disclosed. A method includes obtaining images of objects approaching a door of a property; identifying candidate paths to the door based on the images of the objects approaching the door of the property; determining movement capabilities of the objects; storing the candidate paths to the door labeled by the movement capabilities of the objects that took the paths; determining capability information for a robot at the property that indicates movement capabilities of the robot; selecting, from the candidate paths, a path for the robot to take to the door based on the movement capabilities of the robot and the labels of the candidate paths; and providing guidance information to the robot that guides the robot to the door along the selected path.

First claim

What is claimed is:

1. A method, comprising:
obtaining images of objects approaching a door of a property;
identifying candidate paths to the door based on the images of the objects approaching the door of the property;
determining movement capabilities of the objects;
storing the candidate paths to the door labeled by the movement capabilities of the objects that took the paths;
determining capability information for a robot at the property that indicates movement capabilities of the robot;
selecting, from the candidate paths, a path for the robot to take to the door based on the movement capabilities of the robot and the labels of the candidate paths; and
providing guidance information to the robot that guides the robot to the door along the selected path.

2. The method of claim 1 , wherein determining capability information for the robot at the property that indicates the movement capabilities of the robot comprises:
receiving data indicating a time of arrival of the robot at the property;
accessing a delivery schedule for the property, the delivery schedule including, for one or more scheduled deliveries, an expected time of each delivery and capability information for the robot that is expected to bring each delivery to the property;
determining, based on the delivery schedule, the delivery that corresponds to the time of arrival of the robot; and
based on the delivery that corresponds to the time of arrival of the robot, determining the capability information for the robot.

3. The method of claim 1 , comprising:
accessing an image of the robot;
analyzing the image of the robot;
based on analyzing the image of the robot, determining a characteristic of a package that is being carried by the robot; and
selecting, from the candidate paths, the path for the robot to take to the door based on the movement capabilities of the robot, the characteristic of the package that is being carried by the robot, and the labels of the candidate paths.

4. The method of claim 1 , wherein selecting, from the candidate paths, a path for the robot to take to the door based on the movement capabilities of the robot and the labels of the candidate paths comprises:
based on a rolling capability of the robot, selecting, from the candidate paths that are labeled by the movement capabilities of the objects that took the paths, a path that is associated with a label indicating rolling capabilities of the objects that took the path.

5. The method of claim 1 , wherein selecting, from the candidate paths, a path for the robot to take based on the movement capabilities of the robot and the labels of the candidate paths comprises:
based on a stair climbing capability of the robot, selecting, from the candidate paths that are labeled by the movement capabilities of the objects that took the paths, a path for the robot to take that is associated with a label indicating stair-climbing capabilities of the objects that took the path.

6. The method of claim 1 , wherein determining capability information for the robot at the property that indicates the movement capabilities of the robot comprises:
accessing one or more images of the robot;
analyzing the one or more images of the robot; and
determining the capability information for the robot based on analyzing the one or more images of the robot.

7. The method of claim 6 , wherein analyzing the one or more images of the robot comprises identifying, in the one or more images of the robot, one or more movement mechanisms of the robot.

8. The method of claim 6 , wherein analyzing the one or more images of the robot comprises identifying, in the one or more images of the robot, a type of motion of the robot.

9. The method of claim 1 , comprising:
obtaining images of second objects approaching a drop-off point at the property, the drop-off point being remote from the door of the property;
identifying second candidate paths to the drop-off point based on the images of the second objects approaching the drop-off point;
determining movement capabilities of the second objects; and
storing the second candidate paths to the drop-off point labeled by the movement capabilities of the second objects that took the paths.

10. The method of claim 9 , comprising:
determining capability information for a second robot at the property that indicates movement capabilities of the second robot;
determining, based on the movement capabilities of the second robot, that the second robot is not capable of taking any of the candidate paths to the door of the property;
in response to determining that the second robot is not capable of taking any of the candidate paths to the door of the property, selecting, from the second candidate paths, a path for the second robot to take to the drop-off point based on the movement capabilities of the second robot and the labels of the second candidate paths; and
providing guidance information to the second robot that guides the second robot to the drop-off point along the selected path.

11. The method of claim 9 , wherein the drop-off point is located at a secure area of the property, the method comprising:
receiving credential information for the robot at the property;
based on receiving the credential information for the robot, enabling access for the robot to the secure area of the property;
selecting, from the second candidate paths, a path for the robot to take to the secure area of the property based on the movement capabilities of the robot and the labels of the second candidate paths; and
providing guidance information to the robot that guides the robot to the secure area of the property along the selected path.

12. The method of claim 1 , wherein providing guidance information to the robot comprises:
tracking a location and orientation of the robot as the robot approaches the door; and
providing directions to the robot based on the tracked location and orientation of the robot to guide the robot to the door along the selected path.

13. The method of claim 1 , wherein providing guidance information to the robot comprises providing prescribed directions to the robot to guide the robot to the door along the selected path.

14. The method of claim 1 , wherein identifying candidate paths to the door based on the images of the objects approaching the door of the property comprises:
determining, for each object of the objects approaching the door of the property, a path that the object took to approach the door and a level of success of the object in reaching the door;
determining that the level of success of the object in reaching the door meets success criteria; and
based on determining that the level of success of the object in reaching the door meets success criteria, identifying the path as a candidate path.

15. The method of claim 14 , comprising:
determining, for each object of the objects approaching the door of the property, at least one of a movement mechanism of the object, a type of motion of the object; or a characteristic of a package carried by the object; and
labeling the candidate path with the at least one of the movement mechanism of the object, the type of motion of the object; or the characteristic of the package carried by the object.

16. The method of claim 1 , comprising:
receiving data indicating an environmental condition at the property, and
selecting, from the candidate paths, the path for the robot to take based on the movement capabilities of the robot, the environmental condition at the property, and the labels of the candidate paths.

17. The method of claim

Assignees

• Objectvideo Labs Llc
• Object Video Labs Llc

Inventors

• Madden Donald Gerard
• Shayne Ethan

Classifications

• G05D1/249

  from positioning sensors located off-board the vehicle, e.g. from cameras · CPC title

• G05D1/0246

  using a video camera in combination with image processing means · CPC title

• G07C9/00174

  Electronically operated locks; Circuits therefor; Nonmechanical keys therefor, e.g. passive or active electrical keys or other data carriers without mechanical keys (mechanical locks with electric permutation E05B49/00; arrangements for sensing or reading record carriers G06K7/00; electronic switching H03K17/00) · CPC title

• G06T7/20 Primary

  Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) · CPC title

• G06V40/20

  Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16) · CPC title