Modular robot

Patent metadata

Field	Value
Publication number	US-11548159-B1
Application number	US-201916427317-A
Country	US
Kind code	B1
Filing date	May 30, 2019
Priority date	May 31, 2018
Publication date	Jan 10, 2023
Grant date	Jan 10, 2023

Abstract

Provided is a robot including: a chassis; wheels; electric motors; a network card; sensors; a processor; and a tangible, non-transitory, machine readable medium storing instructions that when executed by the processor effectuates operations including: capturing, with at least one exteroceptive sensor, a first image and a second image; determining, with the processor, an overlapping area of the first image and the second image by comparing the raw pixel intensity values of the first image to the raw pixel intensity values of the second image; combining, with the processor, the first image and the second image at the overlapping area to generate a digital spatial representation of the environment; and estimating, with the processor using a statistical ensemble of simulated positions of the robot, a corrected position of the robot to replace a last known position of the robot within the digital spatial representation of the environment.

First claim

The invention claimed is:

1. A robot, comprising:
a chassis;
a set of wheels coupled to the chassis;
one or more electric motors electrically coupled to the set of wheels;
a network card providing wireless connectivity to the internet;
a plurality of sensors;
a processor electronically coupled to the plurality of sensors and configured to receive sensor readings; and
a tangible, non-transitory, machine readable medium storing instructions that when executed by the processor effectuates operations comprising:
capturing, with at least one exteroceptive sensor of the plurality of sensors, a first image and a second image of an environment of the robot, wherein:
the first image is captured from a first position and orientation and the second image is captured from a second position and orientation, different from the first position and orientation; and
the first image and second image comprise unprocessed raw pixel intensity values or processed depth values;

determining, with the processor, an overlapping area of a field of view of the first image and of a field of view of the second image by comparing the raw pixel intensity values or the processed depth values of the first image to the raw pixel intensity values or depth values of the second image;
combining, with the processor, the first image and the second image at the overlapping area to generate a digital spatial representation of the environment, wherein:
the digital spatial representation of the environment comprises areas of the environment captured within the first image and the second image; and
the digital spatial representation indicates locations of physical objects and boundaries of the environment captured within the first image and the second image; and

estimating, with the processor using a statistical ensemble of simulated locations of the robot within the environment, a corrected location of the robot within the digital spatial representation of the environment to replace a last known location of the robot within the digital spatial representation of the environment, wherein:
the simulated locations of the robot comprise all possible locations within the environment at which the robot can be located;
the statistical ensemble comprises probabilities of the robot being actually located at each of the simulated locations of the robot;
estimating the corrected location of the robot occurs when the processor loses a current location of the robot in the digital spatial representation of the environment during a movement from the last known location to a new intended location while performing a task;
the processor loses the current position of the robot due to a drift in intended trajectory;
and
the processor checks sensor readings received from at least one sensor of the plurality of sensors against each simulated digital spatial representation of the environment corresponding with a perspective of each simulated location of the robot to determine the corrected location of the robot within the digital spatial representation of the environment.

2. The robot of claim 1 , wherein the sensor readings comprise one or more of: visual readings and depth readings captured with the at least one exteroceptive sensor.

3. The robot of claim 1 , wherein the statistical ensemble reduces to at least one location when the sensor readings of the at least one sensor of the plurality of sensors are captured.

4. The robot of claim 1 , wherein the corrected location of the robot is a simulated location of all the simulated locations with the highest probability of being the actual location of the robot based on a best match between the sensor readings from the at least one sensor of the plurality of sensors and a simulated digital spatial representation of the environment corresponding with a perspective of the particular simulated location.

5. The robot of claim 1 , wherein certainty of a location of the robot decreases with movement of the robot and increases with a number of sensor readings captured.

6. The robot of claim 1 , wherein determining the area of overlap of the field of view of the first image and the field of view of the second image comprises aligning the raw pixel intensity values or depth values of the first image and the raw pixel intensity values or depth values of the second image.

7. The robot of claim 6 , wherein aligning the raw pixel intensity values or depth values of the first image and the raw pixel intensity values or depth values of the second image comprises:
determining, with the processor, aggregate amounts of difference between overlapping portions of the first image and the second image at candidate alignments displaced from an approximate alignment; and
selecting, with the processor, the candidate alignment that produces a lowest aggregate amount of difference among the candidate alignments or selecting, with the processor, a candidate alignment that produces an aggregate amount of difference less than a threshold.

8. The robot of claim 1 , wherein the operations further comprise:
determining, with the processor, a movement path of the robot in real-time based on the digital spatial representation of the environment and sensor readings captured with at least some of the plurality of sensors; and
controlling, with the processor, the one or more electric motors of the robot to cause the robot to move along the movement path.

9. The robot of claim 8 , wherein determining the movement path comprises determining lengths of segments of the movement path, the segments having a linear or arc motion trajectory.

10. The robot of claim 8 , wherein the movement path comprises a boustrophedon pattern including segments with motion trajectories in alternating directions.

11. A tangible, non-transitory, machine readable medium storing instructions that when executed by a processor effectuates operations comprising:
capturing, with at least one exteroceptive sensor of the plurality of sensors, a first image and a second image of an environment of the robot, wherein:
the first image is captured from a first position and orientation and the second image is captured from a second position and orientation, different from the first position and orientation; and
the first image and second image comprise unprocessed raw pixel intensity values or processed depth values;

determining, with the processor, an overlapping area of a field of view of the first image and of a field of view of the second image by comparing the raw pixel intensity values or the processed depth values of the first image to the raw pixel intensity values or depth values of the second image;
combining, with the processor, the first image and the second image at the overlapping area to generate a digital spatial representation of the environment, wherein:
the digital spatial representation of the environment comprises areas of the environment captured within the first image and the second image; and
the digital spatial representation indicates locations of physical objects and boundaries of the environment captured within the first image and the second image; and

estimating, with the processor using a statistical ensemble of simulated locations of the robot within the environment, a corrected location of the robot within the digital spatial representation of the environment to replace a last known location of the robot within the digital spatial representation of the environment, wherein:
the simulated locations of the robot comprise all possible locations within the environment at which the robot can be located;
the statistical ensemble comprises probabilities of the robot being actually located at each of the simulated locations o

Assignees

• Ebrahimi Afrouzi Ali
• Fath Lukas
• Zhang Chen
• Highfill Brian
• Ebrahimi Afrouzi Amin
• Ai Inc

Inventors

• Ebrahimi Afrouzi Ali
• Fath Lukas
• Zhang Chen
• Highfill Brian
• Ebrahimi Afrouzi Amin

Classifications

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• B25J9/1697 Primary

  Vision controlled systems · CPC title

• G06T2207/20221

  Image fusion; Image merging · CPC title

• B25J5/007 Primary

  mounted on wheels · CPC title

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title