System for spot cleaning by a mobile robot

Patent metadata

Field	Value
Publication number	US-10575699-B2
Application number	US-201815863705-A
Country	US
Kind code	B2
Filing date	Jan 5, 2018
Priority date	Jan 5, 2018
Publication date	Mar 3, 2020
Grant date	Mar 3, 2020

Abstract

A system for enabling spot cleaning includes a mobile computing device and a mobile cleaning robot. The mobile computing device includes at least one camera configured to capture images of an environment, and at least one data processor configured to (a) establish, based at least in part on first information provided by the at least one image sensor, a coordinate system in the environment, (b) determine, based at least in part on second information provided by the at least one camera, a first set of coordinates of a region at a first location, (c) determine, based at least in part on third information provided by the at least one camera, a second set of coordinates of a mobile cleaning robot at a second location, (d) send the first set of coordinates and second set of coordinates, or coordinates of the first location relative to the second location, to the mobile cleaning robot, and (e) send an instruction to the mobile cleaning robot to request the mobile cleaning robot to travel to the first location.

First claim

What is claimed is:

1. A system for enabling spot cleaning, the system comprising:
a mobile computing device comprising:
at least one camera configured to capture images of an environment;
a storage device storing processor-executable instructions; and
at least one data processor communicatively coupled to the storage device, in which upon execution of the processor-executable instructions by the at least one data processor, the at least one data processor is configured to:
establish, based at least in part on first information provided by the at least one camera of the mobile computing device, a coordinate system corresponding to the environment;
determine, based at least in part on second information provided by the at least one camera of the mobile computing device, a first set of coordinates of a region at a first location;
determine, based at least in part on third information provided by the at least one camera of the mobile computing device, a second set of coordinates of a mobile cleaning robot at a second location;
send at least one of (i) the first set of coordinates and the second set of coordinates, or (ii) the coordinates of the first location relative to the second location, to the mobile cleaning robot; and
send a first instruction to the mobile cleaning robot to request the mobile cleaning robot to travel to the first location.

2. The system of claim 1 in which the at least one data processor is configured to send a second instruction to the mobile cleaning robot to request the mobile cleaning robot to perform a cleaning function in the region at the first location.

3. The system of claim 1 in which the storage device stores a neural network that was trained using images of the mobile cleaning robot or similar mobile cleaning robots, or images derived from the images of the mobile cleaning robot or similar mobile cleaning robots, and the at least one data processor is configured to:
process, using the neural network, images provided by the at least one camera, or images derived from the images of the mobile cleaning robot provided by the at least one camera, to identify the mobile cleaning robot in the images.

4. The system of claim 3 in which the at least one data processor is configured to identify a feature of the mobile cleaning robot in the image, determine coordinates of the feature in the coordinate system, and assign the coordinates of the feature as the second set of coordinates of the mobile cleaning robot.

5. The system of claim 1 in which the at least one data processor is configured to:
use fourth information provided by the at least one camera,
determine an angle of orientation of the mobile cleaning robot, and
send the angle of orientation of the mobile cleaning robot to the mobile cleaning robot.

6. The system of claim 5 in which the at least one data processor is configured to use a neural network to process images of the mobile cleaning robot provided by the at least one camera, or images derived from the images of the mobile cleaning robot provided by the at least one camera, to determine the angle of orientation of the mobile cleaning robot.

7. The system of claim 6 in which the at least one data processor is configured to:
process images of the mobile cleaning robot to generate top-view images of the mobile cleaning robot, and
use the neural network to process the top-view images of the mobile cleaning robot to determine the angle of orientation of the mobile cleaning robot.

8. The system of claim 7 in which the neural network was trained using top-view images of the mobile cleaning robot or similar mobile cleaning robots.

9. The system of claim 5 , further comprising the mobile cleaning robot, in which the mobile cleaning robot is configured to, upon receiving the first instruction, the first set of coordinates, the second set of coordinates, and the angle of orientation, travel from the second location to the first location.

10. The system of claim 1 in which the at least one data processor is configured to:
process video frames provided by the at least one camera and identify feature points in the video frames;
track the feature points across multiple video frames and estimate three-dimensional positions of the feature points;
fit planes to the feature points and identify a plane that best matches the feature points; and
establish the coordinate system based on the identified plane.

11. The system of claim 10 in which the feature points substantially lie on a floor surface, and the identified plane substantially lies on the floor surface.

12. The system of claim 1 in which the storage device stores program code for one or more image detection modules and an augmented reality toolkit, and the at least one data processor is configured to use the one or more image detection modules and the augmented reality toolkit to establish the coordinate system in the environment, determine the first set of coordinates of the region, and determine the second set of coordinates of the mobile cleaning robot.

13. The system of claim 1 in which the first information comprises images of the environment, the second information comprises one or more images of the region at the first location, and the third information comprises one or more images of the mobile cleaning robot.

14. A system for enabling spot cleaning, the system comprising:
a mobile computing device comprising:
at least one camera configured to capture images of an environment;
a storage device storing processor-executable instructions; and
at least one data processor communicatively coupled to the storage device, in which upon execution of the processor-executable instructions by the at least one data processor, the at least one data processor is configured to:
establish, based at least in part on first information provided by the at least one camera of the mobile computing device, a coordinate system corresponding to the environment;
determine, based at least in part on second information provided by the at least one camera of the mobile computing device, a first set of coordinates of a region at a first location;
determine, based at least in part on third information provided by the at least one camera of the mobile computing device, a second set of coordinates of an object in the environment, in which the location of the object is known to a mobile cleaning robot;
send at least one of (i) coordinates of the region and coordinates of the object, or (ii) relative position information about the region and the object, to the mobile cleaning robot; and
send a first instruction to the mobile cleaning robot to request the mobile cleaning robot to travel to the first location.

15. The system of claim 14 , further comprising the mobile cleaning robot, in which the mobile cleaning robot is configured to travel to the first location based on information about the position of the object in a map maintained by the mobile cleaning robot, and information about the position of the first location relative to the object.

16. A method for spot cleaning using a mobile cleaning robot, the method comprising:
using a camera of a mobile computing device to capture images of an environment;
establishing, using at least one data processor of the mobile computing device, a coordinate system corresponding to the environment based at least in part on first information derived from the images of the environment captured using the camera of the mobile computing device;
determining, using the at least one data processor of the mobile computing device, a first set of coordinates of a region at

Assignees

• Irobot Corp

Inventors

• Bassa Angela
• Al-Mohssen Husain

Classifications

• A47L11/4038 Primary

  Disk shaped surface treating tools · CPC title

• G06V30/19173

  Classification techniques · CPC title

• G06V10/82 Primary

  using neural networks · CPC title

• A47L2201/04

  Automatic control of the travelling movement; Automatic obstacle detection · CPC title

• A47L11/4061

  Steering means; Means for avoiding obstacles; Details related to the place where the driver is accommodated · CPC title