Method for estimating distance using point measurement and color depth

Patent metadata

Field	Value
Publication number	US-11468588-B1
Application number	US-202117242020-A
Country	US
Kind code	B1
Filing date	Apr 27, 2021
Priority date	Mar 1, 2016
Publication date	Oct 11, 2022
Grant date	Oct 11, 2022

Abstract

A media storing instructions that when executed by a processor of a robot effectuates operations including detecting an object in a line of sight of at least one sensor; adjusting a current path of the robot to include a detour path around the object, instructing the robot to resume along the current path after avoiding the object, discounting areas of overlap from a total area covered based on at least some data collected by sensors, inferring previously visited areas and unvisited areas, generating a planar representation of a workspace of the robot by stitching data collected by at least some sensors of the robot at overlapping points, and transmitting the planar representation and coverage statistics to an application of a communication device configured to display the information.

First claim

The invention claimed is:

1. A memory storing program code that when executed by a processor of a robot effectuates operations, comprising:
detecting, with a plurality of sensors of the robot, an object in a line of sight of at least one of the plurality of sensors;
adjusting, with the processor of the robot, a current path of the robot to include a detour path around the object, wherein:
the current path comprises at least one segment parallel to a wall;
each of the at least one segment parallel to the wall are positioned at a different distance from the wall and are separated by a distance determined based on a width of a cleaning tool of the robot and a desired amount of overlap; and
each of the at least one segment parallel to the wall are positioned such that areas covered by the robot when driving along two adjacent segments parallel to the wall partially overlap;

instructing, with the processor of the robot, the robot to resume along the current path after avoiding the object;
discounting, with the processor of the robot, areas of overlap from a total area covered based on at least some data collected by sensors of the robot in overlapping areas;
inferring, with the processor of the robot, previously visited areas and unvisited areas;
generating, with the processor of the robot, a planar representation of a workspace of the robot by stitching data collected by at least some sensors of the robot at overlapping points; and
transmitting, with the processor of the robot, at least the planar representation and coverage statistics to an application of a communication device configured to display the planar representation and coverage statistics.

2. The memory of claim 1 , wherein:
the processor uses data from at least one primary sensor type to perform operations; and
the processor uses data from other sensor types to perform operations when the at least one primary sensor type is occluded, obstructed, or malfunctioning.

3. The media of claim 1 , wherein the operations further comprise:
capturing, with a camera of the robot, a plurality of images of the workspace;
inferring, with the processor of the robot, distances to objects in the workspace based on at least some pixels of the plurality of images captured;
determining, with the processor of the robot, an angle of a surface in the plurality of images captured based on at least some of the inferred distances; and
adjusting, with the processor of the robot, a heading angle of the robot relative to the angle of the surface.

4. The memory of claim 1 , wherein the operations further comprise:
marking, with the processor of the robot, a location of detected objects, cliffs, and a charging station within the planar representation; and
determining, with the processor of the robot, a path of the robot based on the location of the detected objects and cliffs.

5. The memory of claim 1 , wherein the operations further comprise at least one of:
actuating, with the processor of the robot, the robot to execute a collaborative task with at least one other robot; and
instruct, with the processor of the robot, at least one other robot to begin performing a task.

6. The memory of claim 1 , wherein the robot is wirelessly connected with the application of the communication device and the application is further configured to:
display at least on of: the planar or a spatial representation of the workspace as its being built and after completion; a movement path of the robot; a current position of the robot; a current position of a charging station of the robot; robot status; a current quantity of total area cleaned; a total area cleaned after completion of a task; a battery level; a current cleaning duration; an estimated total cleaning duration required to complete a task; an estimated total battery power required to complete a task, a time of completion of a task; obstacles within the planar representation including object type of the object and percent confidence of the object type; objects within the planar representation including objects with unidentified object type; issues requiring user attention within the planar representation; a fluid flow rate for different areas within the planar representation; a notification that the robot has reached a particular location; cleaning history; user manual; maintenance information; and firmware information;
receive an input designating at least one of: an object type of an object with unidentified object type; a schedule for cleaning different areas within the planar representation; vacuuming or mopping or vacuuming and mopping for cleaning different areas within the planar representation; a suction level for cleaning different areas within the planar representation; a no-entry zone; a no-mopping zone; a virtual wall; a modification to the planar representation; a fluid flow rate level for mopping different areas within the planar representation; an order of cleaning different areas of the workspace; deletion or addition of a robot paired with the application; an instruction to find the robot; an instruction to contact customer service; an instruction to update firmware; a driving speed of the robot; a volume of the robot; a voice type of the robot; pet details; deletion of an object within the planar representation; an instruction for a charging station of the robot; an instruction for the charging station of the robot to empty a bin of the robot into a bin of the charging station; and an instruction for the charging station of the robot to fill a fluid reservoir of the robot; and
receive an input enacting an instruction for the robot to at least one of: pause a current task; start mopping or vacuuming; dock at the charging station; start cleaning; spot clean; navigate to a particular location; and move or rotate in a particular direction.

7. The memory of claim 1 , wherein the operations further comprise:
determining, with the processor of the robot, a location, a height, a width, and a depth of the object.

8. The memory of claim 1 , wherein the operations further comprise:
determining, with the processor of the robot, an object type of a detected obstacle, wherein the object type includes at least one of cables, cords, wires, toys, jewelry, garments, socks, shoes, shoelaces, feces, liquids, keys, food items, remote controls, plastic bags, purses, backpacks, earphones, cell phones, tablets, laptops, chargers, animals, fridges, televisions, chairs, tables, light fixtures, lamps, fan fixtures, cutlery, dishware, dishwashers, microwaves, coffee makers, smoke alarms, plants, books, washing machines, dryers, watches, blood pressure monitors, blood glucose monitors, first aid items, and Wi-Fi routers; and
storing, with the processor of the robot, an object type of an object identified by a user using the application in an object types dictionary for future recognition.

9. The memory of claim 1 , wherein:
the planar representation comprises two or more rooms of the workspace; and
the operations further comprises:
detecting, with the processor of the robot, a current room;
dividing, with the processor of the robot, the planar representation into rooms and hallways, wherein the rooms and hallways are categorized based on any of: an arrangement of areas within the workspace, visual features recognized, an opening in the planar representation, and input received by the application.

10. The memory of claim 1 , wherein generating the planar representation comprises:
minimizing, with the processor of the robot, a total time or total distance travelled by the robot required to complete the planar representation of the workspace.

11. The memory of claim 1 , further comprising:
perceiving, w

Assignees

• Ebrahimi Afrouzi Ali
• Mehrnia Soroush
• Ai Inc

Inventors

• Ebrahimi Afrouzi Ali
• Mehrnia Soroush

Classifications

• G01C3/085 Primary

  with electronic parallax measurement · CPC title

• G06T7/60 Primary

  Analysis of geometric attributes · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T2207/10024

  Color image · CPC title

• H04N7/185

  from a mobile camera, e.g. for remote control · CPC title