Autonomous gardening vehicle with camera

Patent metadata

Field	Value
Publication number	US-9603300-B2
Application number	US-201414563815-A
Country	US
Kind code	B2
Filing date	Dec 8, 2014
Priority date	Dec 12, 2013
Publication date	Mar 28, 2017
Grant date	Mar 28, 2017

Abstract

Some embodiments described herein include a method for generating scaled terrain information with an unmanned autonomous gardening vehicle. In some embodiments the gardening vehicle includes a driving unit comprising a set of at least one drive wheel and a motor connected to the at least one drive wheel for providing movability of the gardening vehicle, a gardening-tool and a camera for capturing images of a terrain, the camera being positioned and aligned in known manner relative to the gardening vehicle. In context of the method the gardening vehicle is moved in the terrain while concurrently generating a set of image data by capturing an image series of terrain sections so that at least two (successive) images of the image series cover an amount of identical points in the terrain, wherein the terrain sections are defined by a viewing area of the camera at respective positions of the camera while moving.

First claim

What is claimed is:

1. A method for generating scaled terrain information with an unmanned autonomous gardening vehicle, the gardening vehicle comprising
a driving unit comprising
a set of at least one drive wheel, and
a motor connected to the at least one drive wheel for providing movability of the gardening vehicle;

a gardening-tool; and
a camera for capturing images of a terrain, the camera being positioned and aligned in known manner relative to the gardening vehicle;
wherein the method comprises:
moving the gardening vehicle in the terrain whilst concurrently generating a set of image data by capturing an image series of terrain sections so that at least two images of the image series cover an amount of identical points in the terrain, wherein the terrain sections are defined by a viewing area of the camera at respective positions of the camera while moving;
applying a simultaneous localization and mapping (SLAM) algorithm to the set of image data and thereby deriving terrain data, the terrain data comprising:
a three-dimensional point cloud representing the captured terrain, and
position data relating to a relative position of the gardening vehicle in the terrain; and

scaling the three-dimensional point cloud by applying an absolute scale information to the terrain data,
wherein the absolute scale information is derived by
capturing a reference image of a reference body, of which at least one of the appearance and the position in the terrain is known, and
deriving the absolute scale information by image processing based on the appearance of the reference body in the captured reference image and of a known magnification ratio of the camera,
wherein at least one of the dimensions, spatial orientation, and shape of the reference body is pre-known.

2. The method according to claim 1 , wherein the scaling of the point cloud comprises scaling the position data.

3. The method according to claim 1 , wherein the at least two images comprise at least two successive images.

4. The method according to claim 1 , wherein the unmanned autonomous gardening vehicle is implemented as an unmanned autonomous lawnmower, wherein the gardening-tool is implemented as a cutting-tool.

5. The method according to claim 1 , wherein the absolute scale information is derived by receiving movement data from a sensor unit of the gardening vehicle providing information about a distance covered by moving the gardening vehicle, the sensor unit being designed as at least one of
a sensor for detecting revolutions of a wheel of the gardening vehicle;
a distance measuring unit for measuring distances to an object by emitting laser light and receiving the laser light reflected at the object;
a range camera; and
an inertial measuring unit (IMU).

6. The method according to claim 5 , wherein the wheel is one of the at least one drive wheel.

7. The method according to claim 1 , wherein:
transmitting at least one of the set of image data and the terrain data to a database and
storing the data in the database, wherein at least one of:
the gardening vehicle comprises a storing unit comprising the database,
a remote controlling unit for controlling the gardening vehicle comprises the storing unit comprising the database, and
the database is embodied by a data cloud stored on a remote server, wherein the terrain data is derived from the image data by cloud computing based on the simultaneous localization and mapping (SLAM) algorithm and/or the scaling of the point cloud is performed by cloud computing.

8. The method according to claim 1 , wherein a borderline of a working area for the gardening vehicle, within which the gardening vehicle is controllable to autonomously move and work, is defined by at least one of:
teaching the borderline by at least one of:
moving the gardening vehicle along a desired path,
capturing a series of border-images of terrain corresponding to the path and of a defined vicinity relative to the path, the series of border-images represents the image series, and
providing the series of border-images for controlling the gardening vehicle, and

setting the borderline on basis of a terrain map, wherein border-position data is provided for controlling the gardening vehicle.

9. The method according to claim 8 , wherein in case the borderline is defined by teaching,
continuously comparing the border-images of the series of border-images with actual images by image processing, the actual images being captured while moving the gardening vehicle inside the working area,
deriving a rate of matching for every actual image based on the comparison and
controlling the movement of the gardening vehicle based on the rate of matching so that the gardening vehicle automatically moves only inside the working area, providing a movement-controlling command for adapting the movement direction of the gardening vehicle if the rate of matching exceeds a predetermined matching-threshold.

10. The method according to claim 1 , further comprising:
extracting at least one state parameter from at least one of the set of image data and the terrain data which represents an actual state of at least one designated terrain section, the actual state relating to a state of at least one plant and ground,
comparing the at least one state parameter to a predetermined threshold for a respective state and
deriving gardener information based on the comparison of the predetermined threshold and the at least one state parameter.

11. The method according to claim 10 , wherein the at least one state parameter provides at least one terrain factor of a group of terrain factors, the group of terrain factors comprising at least one of:
plant height,
plant growth,
humidity of the terrain,
density of plants,
planarity of the terrain and
brightness or colour of the terrain.

12. The method according to claim 10 , wherein at least one of:
the gardener information is provided to a user of the gardening vehicle together with a related recommendation concerning a suggested treatment of the respective at least one designated terrain section, and
the gardening-tool is applied based on the gardener information.

13. The method according to claim 12 , wherein the gardening tool comprises at least one of a hedge-cutter, a tree-branch cutter, a grass-cutter, scissors, a fertilising unit, a pesticide unit, a watering unit, and a lawn thatcher.

14. The method according to claim 1 , further comprising:
applying the gardening-tool based on the terrain data, in particular based on the point cloud, wherein
the terrain data represents at least an actual shape of an object in the captured terrain,
gardening data is provided representing a target shape for the object and
the gardening-tool is guided based on the gardening data so that the actual shape of the object is transferred into the target shape,

wherein the gardening vehicle is positioned at a designated position in the terrain based on the terrain data and the gardening-tool is guided according to a designated shape of a plant based on the terrain data and the gardening data.

15. The method according to claim 1 , further comprising at least one of:
providing controlling information for controlling a further gardening unit in the terrain, wherein the controlling information is derived by the terrain data and an actual position and orientation of the further gardening unit in the terrain, wherein the position and orientation of the gardening unit is derived from the terrain data by data processing or by image processing of an image covering at least a part of the gardening un

Assignees

• Hexagon Technology Ct Gmbh

Inventors

• Pettersson Bo

Classifications

• G05D1/0246

  using a video camera in combination with image processing means · CPC title

• G05D1/0251

  extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision (stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593) · CPC title

• A01D34/008 Primary

  for automated or remotely controlled operation · CPC title

• G05D2201/0208

  Physics · mapped topic

• A01G25/09

  Watering arrangements making use of movable installations on wheels or the like · CPC title