Local area mapping for a robot lawnmower

Patent metadata

Field	Value
Publication number	US-12153434-B2
Application number	US-202217835697-A
Country	US
Kind code	B2
Filing date	Jun 8, 2022
Priority date	Jun 8, 2022
Publication date	Nov 26, 2024
Grant date	Nov 26, 2024

Abstract

A robot lawnmower and method for controlling the robot lawnmower based on the generation of a local map. The method including receiving an image from an imaging sensor onboard the robot lawnmower, the image including an area of ground in an upcoming path, applying a semantic segmentation algorithm to produce a segmented image from the received image, the segmented image including regions corresponding to features in the image, applying a perspective transform to the segmented image to obtain an overhead view transformed image, wherein the regions are preserved in the transformed image, determining, from the transformed image, positions of the regions relative to the current position of the robot lawnmower, plotting a local map of the environment of the robot lawnmower based on positions of the regions relative to a current position of the robot lawnmower; and controlling the robot lawnmower to navigate a lawn area using the local map.

First claim

The invention claimed is:

1. A computer-implemented method of controlling a robot lawnmower, the method comprising:
receiving an image from an imaging sensor on-board the robot lawnmower, the image including an area of ground in an upcoming path of the robot lawnmower;
applying a semantic segmentation algorithm to produce a segmented image from the received image, the segmented image including regions corresponding to features in the received image;
applying a perspective transform to the segmented image to obtain an overhead-view transformed image, wherein the regions are preserved in the overhead-view transformed image;
determining, from the overhead-view transformed image, positions of the regions relative to a current position of the robot lawnmower;
plotting a local map of an environment of the robot lawnmower based on the positions of the regions relative to the current position of the robot lawnmower, wherein plotting the local map includes:
obtaining a previous iteration of the local map, plotted using data from previously received images;
adding, to the previous iteration of the local map, one or more first regions, from the overhead-view transformed image corresponding to the received image, having a distance from the current position of the robot lawnmower that is less than a threshold distance; and
deleting a portion of the previous iteration of the local map corresponding to one or more second regions having a distance from the current position of the robot lawnmower that is greater than the threshold distance; and

controlling the robot lawnmower to navigate a lawn area using the local map.

2. The method of claim 1 , wherein the regions relative to the current position of the robot lawnmower include one or more third regions including non-grass features, and wherein the non-grass features include one or more of an obstacle, hazard or boundary of a lawn area.

3. The method of claim 1 , wherein the local map is size-limited according to the threshold distance from the current position of the robot lawnmower.

4. The method of claim 1 , wherein plotting the local map further includes:
recording a number of previous images used to plot the previous iteration of the local map;
comparing the recorded number of images to a maximum image number; and
when the recorded number of images used to plot the previous iteration of the local map is equal to the maximum image number:
upon receiving the image from the imaging sensor, deleting a portion of the previous iteration of the local map corresponding to an oldest of the previously received images; and
adding, to the previous iteration of the local map, one or more third regions, from the overhead-view transformed image corresponding to the received image, such that the maximum image number is not exceeded.

5. The method of claim 1 , wherein determining, from the overhead-view transformed image, the positions of the regions relative to the current position of the robot lawnmower includes:
processing the overhead-view transformed image with a VSLAM algorithm to obtain the positions of the regions of the overhead-view transformed image, relative to the current position of the robot lawnmower.

6. The method of claim 1 , further comprising:
obtaining additional data relating to a motion of the robot lawnmower; and
updating the positions of the regions in the local map relative to the current position of the robot lawnmower based on the additional data, wherein the additional data includes sensor data from one or more additional sensors, and wherein the sensor data includes one or more of: odometry data, IMU data, or GPS data.

7. The method of claim 6 , wherein updating the positions of the regions in the local map relative to the current position of the robot lawnmower based on the additional data comprises:
for each region in the local map:
obtaining a previous position of a respective region from the local map; and
modifying the previous position of the respective region based on the additional sensor data to determine an updated position of the respective region.

8. The method of claim 6 , wherein the additional data includes region tracking data for one or more third regions present in the received image.

9. The method of claim 8 , further comprising:
generating the region tracking data for the one or more third regions present in the received image, by:
identifying the one or more third regions in one or more previously received images;
tracking the one or more third regions through the one or more previously received images and the received image to determine a tracking path for each of the one or more fourth regions.

10. The method of claim 9 , further comprising:
extrapolating the tracking path of the one or more third regions when the one or more third regions are no longer present in a subsequently received image; and
updating one or more corresponding positions of the one or more third regions in the local map relative to the current position of the robot lawnmower based on the extrapolated tracking path.

11. The method of claim 6 , wherein updating the positions of the regions in the local map relative to the current position of the robot lawnmower based on the additional data is performed at a higher frequency than updating the positions of the regions in the local map relative to the current position of the robot lawnmower based on the received image.

12. The method of claim 1 , wherein controlling the robot lawnmower to navigate the lawn area using the local map comprises:
accessing the local map; and
controlling the robot lawnmower to navigate the lawn area according to the positions of the regions indicated by the local map.

13. The method of claim 12 , wherein controlling the robot lawnmower to navigate the lawn area according to the positions of the regions indicated by the local map comprises:
controlling one or more actuation mechanisms of the robot lawnmower to cause the robot lawnmower to move to navigate the lawn area.

14. A method comprising:
receiving an image from an imaging sensor on-board a robot lawnmower, the image including an area of ground in an upcoming path of the robot lawnmower;
applying a semantic segmentation algorithm to produce a segmented image from the received image, the segmented image including regions corresponding to features in the received image;
applying a perspective transform to the segmented image to obtain an overhead-view transformed image, wherein the regions are preserved in the overhead-view transformed image;
determining, from the overhead-view transformed image, positions of the regions relative to a current position of the robot lawnmower;
plotting a local map of an environment of the robot lawnmower based on the positions of the regions relative to the current position of the robot lawnmower, wherein plotting the local map includes:
obtaining a previous iteration of the local map, plotted using data from previously received images; and
adding, to the previous iteration of the local map, one or more first regions, from the overhead-view transformed image corresponding to the received image;
recording a number of previous images used to plot the previous iteration of the local map;
comparing the recorded number of images to a maximum image number; and
when the recorded number of images used to plot the previous iteration of the local map is equal to the maximum image number:
upon receiving the image from the imaging sensor, deleting a portion of the previous iteration of the local map corresponding to an oldest of the previous images, wherein adding, to the previous iteration o

Assignees

• Positec Power Tools Suzhou Co Ltd

Inventors

• Hoffman John

Classifications

• G05D1/246

  using environment maps, e.g. simultaneous localisation and mapping [SLAM] · CPC title

• G05D1/617

  Safety or protection, e.g. defining protection zones around obstacles or avoiding hazards (arrangements for controlling the position or course of two or more vehicles for avoiding collisions therebetween G05D1/693; arrangements for reacting to or preventing system or operator failure G05D1/80) · CPC title

• G05D1/648

  Performing a task within a working area or space, e.g. cleaning · CPC title

• G05D1/249

  from positioning sensors located off-board the vehicle, e.g. from cameras · CPC title

• A01D2101/00

  Lawn-mowers · CPC title