Local area mapping for a robot lawnmower

Patent metadata

Field	Value
Publication number	US-2023400857-A1
Application number	US-202217835697-A
Country	US
Kind code	A1
Filing date	Jun 8, 2022
Priority date	Jun 8, 2022
Publication date	Dec 14, 2023
Grant date	—

Abstract

A robot lawnmower and method for controlling the robot lawnmower based on the generation of a local map. The method including receiving an image from an imaging sensor onboard the robot lawnmower, the image including an area of ground in an upcoming path, applying a semantic segmentation algorithm to produce a segmented image from the received image, the segmented image including regions corresponding to features in the image, applying a perspective transform to the segmented image to obtain an overhead view transformed image, wherein the regions are preserved in the transformed image, determining, from the transformed image, positions of the regions relative to the current position of the robot lawnmower, plotting a local map of the environment of the robot lawnmower based on positions of the regions relative to a current position of the robot lawnmower; and controlling the robot lawnmower to navigate a lawn area using the local map.

First claim

1 . A computer-implemented method of controlling a robot lawnmower, the method comprising:
receiving an image from an imaging sensor on-board a robot lawnmower, the image including an area of ground in an upcoming path of the robot lawnmower; applying a semantic segmentation algorithm to produce a segmented image from the received image, the segmented image including regions corresponding to features in the image; applying a perspective transform to the segmented image to obtain an overhead-view transformed image, wherein the regions are preserved in the transformed image; determining, from the transformed image, positions of the regions relative to the current position of the robot lawnmower; plotting a local map of the environment of the robot lawnmower based on positions of the regions relative to a current position of the robot lawnmower; and controlling the robot lawnmower to navigate a lawn area using the local map.

2 . The method of claim 1 , wherein the regions include one or more regions including non-grass features, and
wherein the non-grass features include one or more of an obstacle, hazard and/or boundary of a lawn area.

3 . The method of claim 1 , wherein the local map is size-limited according to a threshold distance from the current position of the robot lawnmower, such that the local map includes regions that are a distance from the current position of the robot lawnmower that is below the threshold distance.

4 . The method of claim 1 , wherein plotting the local map includes:
obtaining a previous iteration of the local map, plotted using data from previously received images; and adding, to the previous iteration of the local map, one or more regions, from the transformed image corresponding to the received image.

5 . The method of claim 4 , wherein plotting the local map includes:
deleting a portion of the previous iteration of the local map corresponding to one or more regions that are a distance from the current position of the robot lawnmower that is greater than a threshold distance; and adding, to the previous iteration of the local map, one or more regions, from the transformed image corresponding to the received image, having a distance from the current position of the robot lawnmower that is less than the threshold distance.

6 . The method of claim 4 , wherein plotting the local map includes:
recording a number of previous images used to plot the previous iteration of the local map; comparing the recorded number of images to a maximum image number; and when the recorded number of images used to plot the previous iteration of the local map is equal to the maximum image number: upon receiving the image from the imaging sensor, deleting a portion of the previous iteration of the local map corresponding to an oldest of the previous images; and adding, to the previous iteration of the local map, one or more regions, from the transformed image corresponding to the received image, such that the local map the maximum image number is not exceeded.

7 . The method of claim 1 , wherein determining, from the transformed image, the positions of the regions relative to the current position of the robot lawnmower includes:
processing the transformed image with a VSLAM algorithm to obtain the positions of the regions of the transformed image, relative to the current position of the robot lawnmower.

8 . The method of claim 1 , further comprising:
obtaining additional data relating to the motion of the robot lawnmower; and updating the positions of the regions in the local map relative to the current position of the robot lawnmower based on the additional data,
wherein the additional data includes sensor data from one or more additional sensors, and
wherein the sensor data includes one or more of:
odometry data,
IMU data, and
GPS data.

9 . The method of claim 8 , wherein updating the positions of the regions in the local map relative to the current position of the robot lawnmower based on the additional data comprises:
for each region in the local map: obtaining a previous position of the region from the local map; and modifying the previous position of the region based on the additional sensor data to determine an updated position of the region.

10 . The method of claim 8 , wherein the additional data includes region tracking data for one or more regions present in the received image.

11 . The method of claim 10 , further comprising:
generating the region tracking data for the one or more regions present in the received image, by:
identifying the one or more regions in one or more previously received images;
tracking the one or more regions through the one or more previously received images and the received image to determine a tracking path for each of the one or more regions.

12 . The method of claim 11 , further comprising:
extrapolating the tracking path of the one or more regions when the one or more regions are no longer present in a subsequently received image; and
updating the position of the one or more regions in the local map relative to the current position of the robot lawnmower based on the extrapolated tracking path.

13 . The method of claim 8 , wherein updating positions of the regions in the local map relative to the current position of the robot lawnmower based on the additional data is performed at a higher frequency than updating positions of the regions in the local map relative to the current position of the robot lawnmower based on the received image.

14 . The method of claim 1 , wherein controlling the robot lawnmower to navigate a lawn area using the local map comprises:
accessing the local map; and controlling the robot lawnmower to navigate the lawn area according to the positions of the regions indicated by the local map.

15 . The method of claim 14 , wherein controlling the robot lawnmower to navigate the lawn area according to the positions of the regions indicated by the local map comprises:
controlling one or more actuation mechanisms of the robot lawnmower to cause the robot lawnmower to move to navigate the lawn area.

16 . A robot lawnmower, comprising:
one or more actuation mechanisms; an imaging sensor; a memory module; and a control module including a processor communicatively coupled to the one or more actuation mechanisms, the imaging sensor, and the memory module; the processor being configured to perform the method of claim 1 .

17 . The robot lawnmower of claim 16 , wherein the imaging sensor is a camera configured to capture wide-angle images using a field of view of over 100 degrees.

18 . The robot lawnmower of claim 16 , further comprising:
one or more additional sensors communicatively coupled to the processor, the one or more additional sensors being configured to provide sensor data relating to at least one of: a speed of the robot lawnmower; a bearing of the robot lawnmower; an orientation of the robot lawnmower; a distance travelled by the robot lawnmower; an acceleration of the robot lawnmower; and a distance from the robot lawnmower to one or more obstacles in the surroundings of the robot lawnmower.

19 . The robot lawnmower of claim 16 , wherein the control module is configured to communicate with an external server.

20 . A computer readable medium having instructions stored thereon, which, when executed by a processor, cause the processor to perform the method of claim 1 .

Assignees

• Positec Power Tools Suzhou Co Ltd

Inventors

• Hoffman John

Classifications

• G05D1/246

  using environment maps, e.g. simultaneous localisation and mapping [SLAM] · CPC title

• G05D1/617

  Safety or protection, e.g. defining protection zones around obstacles or avoiding hazards (arrangements for controlling the position or course of two or more vehicles for avoiding collisions therebetween G05D1/693; arrangements for reacting to or preventing system or operator failure G05D1/80) · CPC title

• G05D1/648

  Performing a task within a working area or space, e.g. cleaning · CPC title

• G05D1/249

  from positioning sensors located off-board the vehicle, e.g. from cameras · CPC title

• G05D1/0214 Primary

  in accordance with safety or protection criteria, e.g. avoiding hazardous areas (monitoring the location of vehicles within a certain area, e.g. forbidden or allowed areas, in traffic control systems for road vehicles G08G1/13) · CPC title