Method for estimating distance using point measurement and color depth

Patent metadata

Field	Value
Publication number	US-11037320-B1
Application number	US-202016932495-A
Country	US
Kind code	B1
Filing date	Jul 17, 2020
Priority date	Mar 1, 2016
Publication date	Jun 15, 2021
Grant date	Jun 15, 2021

Abstract

A method including detecting an object in a line of sight of at least one sensor; adjusting a current path of the robot to include a detour path around the object, instructing the robot to resume along the current path after avoiding the object, discarding at least some data collected by sensors of the robot in overlapping areas covered, inferring previously visited areas and unvisited areas, generating a planar representation of a workspace of the robot by stitching data collected by at least some sensors of the robot at overlapping points, and presenting at least the planar representation and coverage statistics on an application of a communication device.

First claim

The invention claimed is:

1. A method for coverage by a robot, comprising:
detecting, with a plurality of sensors of the robot, an object in a line of sight of at least one of the plurality of sensors;
adjusting, with a processor of the robot, a current path of the robot to include a detour path around the object, wherein:
the current path comprises at least one segment parallel to a wall;
each of the at least one segment parallel to the wall is positioned a predetermined distance from the wall;
each of the at least one segment parallel to the wall is positioned such that areas covered by the robot when driving along two adjacent segments parallel to the wall partially overlap;

instructing, with the processor of the robot, the robot to resume along the current path after avoiding the object;
discarding, with the processor of the robot, at least some data collected by sensors of the robot in overlapping areas covered;
inferring, with the processor of the robot, previously visited areas and unvisited areas;
generating, with the processor of the robot, a planar representation of a workspace of the robot by stitching data collected by at least some sensors of the robot at overlapping points; and
presenting, with an application of a communication device, at least the planar representation and coverage statistics.

2. The method of claim 1 , wherein:
the processor uses data from at least one primary sensor type to perform operations; and
the processor uses data from other sensor types to perform operations when the at least one primary sensor type is occluded, obstructed, or malfunctioning.

3. The method of claim 1 , further comprising:
capturing, with a camera of the robot, a plurality of images of the workspace;
inferring, with the processor of the robot, distances to objects in the workspace based on at least some pixels of the plurality of images captured;
determining, with the processor of the robot, an angle of a surface in the plurality of images captured based on at least some of the inferred distances; and
adjusting, with the processor of the robot, a heading angle of the robot relative to the angle of the surface.

4. The method of claim 1 , wherein the planar representation is based on at least inferred distances to objects within the workspace.

5. The method of claim 1 , further comprising:
marking, with the processor of the robot, a location of detected objects within the planar representation; and
determining, with the processor of the robot, a path of the robot based on the location of the detected objects.

6. The method of claim 1 , further comprising:
detecting, with a floor sensor of the robot, a cliff within the workspace;
marking, with the processor of the robot, a location of detected cliffs within the planar representation; and
determining, with the processor of the robot, a path of the robot based on the location of the detected cliffs.

7. The method of claim 1 , wherein the robot comprises at least a camera and a depth sensor.

8. The method of claim 1 , wherein the robot patrols at least one area of the workspace or drives from a first point to a second point.

9. The method of claim 1 , wherein the at least one segment of the current path is the longest possible segment that is parallel to a wall within the workspace.

10. The method of claim 1 , wherein at least a portion of processing operations are offloaded from the robot to the cloud.

11. The method of claim 1 , wherein the robot is configured to at least one of: execute a collaborative task with at least one other robot and instruct at least one other robot to begin performing a task.

12. The method of claim 1 , wherein the robot is wirelessly connected with an application of a communication device.

13. The method of claim 1 , wherein the application is configured to:
display at least on of: the planar or a spatial representation of the workspace as its being built and after completion; a movement path of the robot; a current position of the robot; a current position of a charging station of the robot; robot status; a current quantity of total area cleaned; a total area cleaned after completion of a task; a battery level; a current cleaning duration; an estimated total cleaning duration required to complete a task; an estimated total battery power required to complete a task, a time of completion of a task; obstacles within the planar representation including object type of the object and percent confidence of the object type; objects within the planar representation including objects with unidentified object type; issues requiring user attention within the planar representation; a fluid flow rate for different areas within the planar representation; a notification that the robot has reached a particular location; cleaning history; user manual; maintenance information; and firmware information;
receive an input designating at least one of: an object type of an object with unidentified object type; a schedule for cleaning different areas within the planar representation; vacuuming or mopping or vacuuming and mopping for cleaning different areas within the planar representation; a suction level for cleaning different areas within the planar representation; a no-entry zone; a no-mopping zone; a virtual wall; a modification to the planar representation; a fluid flow rate level for mopping different areas within the planar representation; an order of cleaning different areas of the workspace; deletion or addition of a robot paired with the application; an instruction to find the robot; an instruction to contact customer service; an instruction to update firmware; a driving speed of the robot; a volume of the robot; a voice type of the robot; pet details; deletion of an object within the planar representation; an instruction for a charging station of the robot; an instruction for the charging station of the robot to empty a bin of the robot into a bin of the charging station; and an instruction for the charging station of the robot to fill a fluid reservoir of the robot; and
receive an input enacting an instruction for the robot to at least one of: pause a current task; start mopping or vacuuming; dock at the charging station; start cleaning; spot clean; navigate to a particular location; and move or rotate in a particular direction.

14. The method of claim 1 , further comprising:
determining, with the processor of the robot, a location, a height, a width, and a depth of the object.

15. The method of claim 1 , further comprising:
determining, with the processor of the robot, an object type of a detected obstacle, wherein the object type includes at least one of cables, cords, wires, toys, jewelry, garments, socks, shoes, shoelaces, feces, liquids, keys, food items, remote controls, plastic bags, purses, backpacks, earphones, cell phones, tablets, laptops, chargers, animals, fridges, televisions, chairs, tables, light fixtures, lamps, fan fixtures, cutlery, dishware, dishwashers, microwaves, coffee makers, smoke alarms, plants, books, washing machines, dryers, watches, blood pressure monitors, blood glucose monitors, first aid items, and Wi-Fi routers; and
storing, with the processor of the robot, an object type of an object identified by a user using the application in an object types dictionary for future recognition.

16. The method of claim 1 , wherein:
the planar representation comprises two or more rooms of the workspace; and
the method further comprises:
detecting, with the processor of the robot, a current room;
dividing, with the processor of the robot, the planar representation into ro

Assignees

• Ebrahimi Afrouzi Ali
• Mehrnia Soroush
• Ai Inc

Inventors

• Ebrahimi Afrouzi Ali
• Mehrnia Soroush

Classifications

• G06T7/60 Primary

  Analysis of geometric attributes · CPC title

• H04N7/185

  from a mobile camera, e.g. for remote control · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T2207/10024

  Color image · CPC title

• G06T7/521

  from laser ranging, e.g. using interferometry; from the projection of structured light · CPC title