Method and control system for surveying and mapping a terrain while operating a bulldozer

Patent metadata

Field	Value
Publication number	US-9481982-B1
Application number	US-201615133145-A
Country	US
Kind code	B1
Filing date	Apr 19, 2016
Priority date	Apr 21, 2015
Publication date	Nov 1, 2016
Grant date	Nov 1, 2016

Abstract

A Method for generating scaled terrain information while operating a bulldozer. The bulldozer may include a driving unit comprising a set of drive wheels, a motor connected to at least one of the drive wheels, a blade for altering the surface of the terrain, at least one camera for capturing images of the environment, the camera being positioned and aligned in a known manner relative to the bulldozer, and a controlling and processing unit. A method may include moving the bulldozer while concurrently generating a set of image data by capturing an image series of terrain sections with the at least one camera so that at least two images of the image series cover an amount of identical points in the terrain, and either applying a simultaneous localization and mapping (SLAM) algorithm or a stereo photogrammetry algorithm to the set of image data and thereby deriving terrain data.

First claim

What is claimed is:

1. A method for generating scaled terrain information while operating a bulldozer, the bulldozer comprising
a driving unit comprising:
a set of drive wheels being part of a continuous track, and
a motor connected to at least one of the drive wheels for providing movability of the bulldozer,

a blade for altering the surface of the terrain,
at least one camera for capturing images of the environment, the camera being positioned and aligned in a known manner relative to the bulldozer, and
a controlling and processing unit,
wherein the method comprises:
moving the bulldozer in the terrain whilst concurrently generating a set of image data by capturing an image series of terrain sections with the at least one camera so that at least two successive images of the image series cover an amount of identical points in the terrain, wherein the terrain sections are defined by a viewing field of the at least one camera at respective positions of the camera while moving,
and either applying
a simultaneous localisation and mapping (SLAM) algorithm or
a stereo photogrammetry algorithm
to the set of image data and thereby deriving terrain data, the terrain data comprising
a point cloud representing the captured terrain and
position data relating to a relative position of the bulldozer in the terrain, and
scaling the point cloud by applying an absolute scale information to the terrain data.

2. The method according to claim 1 , wherein:
the absolute scale information is derived by receiving movement data from a sensor unit of the bulldozer providing information about a distance covered by moving the bulldozer, the sensor unit being designed as
a sensor for detecting revolutions of at least one of the wheels of the bulldozer, and/or
a distance measuring unit for measuring distances to an object by emitting laser light and receiving the laser light reflected at the object, and/or
a range camera, and/or
an inertial measuring unit (IMU), and/or
a receiving unit for global navigation satellite system (GNSS) signals and/or
differential global navigation satellite system (DGNSS) signals.

3. The method according to claim 1 , wherein:
the absolute scale information is derived by capturing a reference image of a reference body of known appearance and/or position in the terrain and deriving the absolute scale information by image processing based on the appearance of the reference body in the captured reference image and of a known magnification ratio of the camera, wherein the dimensions, spatial orientation and/or shape of the reference body is pre-known.

4. The method according to claim 1 , wherein:
transmitting the set of image data and/or the terrain data to a database and storing the data in it, wherein
the bulldozer comprises a storing unit comprising the database, and/or
a remote controlling unit for controlling the bulldozer comprises the storing unit comprising the database, and/or
the database is embodied by a data cloud stored on a remote server, wherein
the terrain data is derived from the image data by cloud computing based on the simultaneous localisation and mapping (SLAM) algorithm or the stereo photogrammetry algorithm, and/or
the scaling of the point cloud is performed by cloud computing.

5. The method according to claim 1 , wherein:
transmitting the image data and/or the terrain data to other working machines located in the terrain and/or
receiving image data and/or the terrain data from other working machines located in the terrain
via a network provided by the working machines, wherein each working machine is configured to:
to process the image data and/or terrain data for updating the current terrain state, and/or
to transmit the image data and/or terrain data to a database, and/or
to transmit and/or to reference position data relating to a relative position of the other working machines in the terrain.

6. The method according to claim 1 , wherein:
extracting at least one state parameter from the set of image data and/or from the terrain data which represents an actual state of at least one designated terrain section,
comparing the state parameter to a predetermined threshold for the respective state, and
deriving constructional information based on the comparison of the predetermined threshold and the state parameter,
wherein the state parameter provides at least one terrain factor of a group of terrain factors, the group of terrain factors comprising at least one or more of the factors selected from the list consisting of:
granularity of the terrain,
substance of the terrain,
brightness or colour of the terrain,
humidity of the terrain,
planarity of the terrain,
volume of the terrain, and
height of the terrain.

7. The method according to claim 6 , wherein:
the constructional information is provided to a user of the bulldozer, together with a related recommendation concerning a suggested bulldozing of the respective at least one designated terrain section.

8. The method according to claim 1 , wherein:
moving the bulldozer and/or blade, based on the terrain data, wherein
the terrain data represents at least an actual shape of the captured terrain,
constructional data is provided representing a target shape for the terrain and
the blade is guided based on the constructional data so that the actual shape of the terrain is transferred into the target shape,
wherein the bulldozer is positioned at a designated position in the terrain based on the terrain data and the blade is guided according to a designated shape of a terrain based on the terrain data and the constructional data.

9. The method according to claim 1 , wherein:
creating a digital terrain map is based on the point cloud.

10. The method according to claim 1 , wherein:
creating a digital terrain map includes creating a digital terrain 3D-model.

11. The method according to claim 1 , wherein:
creating a digital terrain map is based on the terrain data gathered by other working machines and obtained via the network.

12. The method according to claim 1 , wherein recognizing objects constituting an obstacle in the proposed path of the bulldozer, wherein a collision of the bulldozer with a recognized object is automatically avoided.

13. The method according to claim 1 , wherein recognizing objects to be excluded or subtracted from the digital terrain map.

14. The method according to claim 1 , wherein recognizing the current state of the terrain sections, which remain to receive a treatment to be provided by at least one other working machine, and sending out a signal via the network to the at least one other working machine, directing the at least one other working machine to fulfill the corresponding treatment in the respective terrain section.

15. A control system for use with a bulldozer, the bulldozer comprising:
a driving unit comprising
a set of drive wheels, and
a motor connected to at least one of the drive wheels for providing movability of the bulldozer and
a blade for altering the surface of the terrain, and
the control system comprising at least
one camera for capturing images of a terrain, the camera to be positioned and aligned in a known manner relative to the bulldozer, and
a controlling and processing unit, wherein:
the control system provides a functionality adapted to generate scaled terrain information by executing, respectively controlling, following steps:
concurrently generating a set of image data by capturing an image series of terrain sections with the camera whilst moving the bulldozer in the terrain so that at least two

Assignees

• Hexagon Technology Ct Gmbh

Inventors

• Pettersson Bo
• Zebhauser Benedikt

Classifications

• E02F9/262 Primary

  with follow-up actions to control the work tool, e.g. controller · CPC title

• G01S17/08

  for measuring distance only (indirect measurement G01S17/46; active triangulation systems G01S17/48) · CPC title

• E02F3/844

  for positioning the blade, e.g. hydraulically · CPC title

• G06K9/00805

  Physics · mapped topic

• E02F3/841

  Devices for controlling and guiding the whole machine, e.g. by feeler elements and reference lines placed exteriorly of the machine (construction of roads E01C19/008) · CPC title