Road-terrain detection method and system for driver assistance systems

Patent metadata

Field	Value
Publication number	US-9435885-B2
Application number	US-201213558407-A
Country	US
Kind code	B2
Filing date	Jul 26, 2012
Priority date	Sep 28, 2011
Publication date	Sep 6, 2016
Grant date	Sep 6, 2016

Abstract

The present invention describes a road terrain detection system that comprises a method for classifying selected locations in the environment of a vehicle based on sensory input signals such as pixel values of a camera image. The method comprises a high level spatial feature generation for selected locations in the environment called base points. The spatial feature generation of the base points is based on a value-continuous confidence representation that captures visual and physical properties of the environment, generated by so called base classifiers operating on raw sensory data. Consequently, the road terrain detection incorporates both local properties of sensor data and their spatial relationship in a two-step feature extraction process.

First claim

The invention claimed is:

1. A road-terrain detection method for driver assistance systems, wherein the road-terrain detection method comprises
sensing the environment of a vehicle with at least one sensor;
transforming a sensor signal from the at least one sensor into at least one confidence map of local properties of the environment by using at least one base classifier;
generating, for base points on the at least one confidence map, each reflecting a specific location in the environment of the vehicle, spatial features for the local properties based on the at least one confidence map; and
classifying, based on the generated spatial features, locations in the environment of the vehicle to a certain category of road terrain corresponding to a type of surface of a road, wherein the generation of a spatial feature for a specific location in the environment of the vehicle comprises
extracting a ray, wherein a ray is defined as a directed line with a certain angular orientation starting from the location in the environment, and
analyzing confidence values along the ray to extract spatial feature.

2. The method according to claim 1 , wherein the at least one sensor is one or more cameras, radars, laser scanners or GPS\navigation systems.

3. The method according to claim 1 , wherein based on the classified locations in the environment a visual or acoustic signal is generated or an effector of the vehicle is operated.

4. The method according to claim 3 , wherein the effector comprises at least one of a steering wheel, an accelerator, or a brake.

5. The method according to claim 1 , wherein the method applied for a specific road terrain is automatically parameterized by using positive and negative samples, which are given by training regions.

6. The method according to claim 5 , wherein the specific road terrain term comprises at least one of “road-like area”, “drivable road”, “ego-lane”, “non-ego-lane”, “non-drivable road”, “sidewalk”, “traffic island”, or “off-limits terrain”.

7. The method according to claim 5 , wherein the training regions comprise polygons.

8. The method according to claim 1 comprising a step of sensory preprocessing for all sensor signals to obtain a common representation of the environment.

9. The method according to claim 8 , wherein the sensory preprocessing is an inverse perspective mapping of a camera image to get a metric image representation.

10. The method according to claim 1 , wherein the analysis of the confidence values along the ray is performed by
integrating the confidence values along the ray;
extracting the ray length, at which the integral exceeds an absorption threshold, which is a certain numeric value.

11. The method according to claim 10 , wherein for a specific location in the environment at least one ray and at least one absorption threshold is used to generate a feature vector that encodes the relative position to a local property given by a confidence map.

12. The method according to claim 1 comprising a base classifier for road classification that generates a confidence map, wherein locations in the environment that correspond to a road-like area have high confidence values.

13. The method according to claim 1 comprising a base classifier for lane marking detection that generates a confidence map, wherein locations in the environment that correspond to lane markings have high confidence values.

14. A road-terrain detection system for driver assistance systems, the road-terrain detection system being adapted to perform a method according to claim 1 .

15. A road-terrain detection system for driver assistance systems, the road-terrain detection system being adapted to
sense with at least one sensor the environment of a vehicle;
transform at least one sensor signal from at least one sensor into at least one confidence map of local properties of the environment by using at least one base classifier;
generate, for base points on the at least one confidence map, each reflecting a specific location in the environment of the vehicle, spatial features for local properties based on the at least one confidence map; and
classify, based on the generated spatial features, locations in the environment of the vehicle to a certain category of road terrain corresponding to a type of surface of a road, wherein the generation of a spatial feature for a specific location in the environment of the vehicle comprises
extracting a ray, wherein a ray is defined as a directed line with a certain angular orientation starting from the location in the environment, and
analyzing confidence values along the ray to extract spatial feature.

16. The system according to claim 15 , wherein the road terrain detection system can select one out of multiple road terrain classification outputs as a system output, in order to internally have different subsystems for coping with
different weather conditions resulting in different visual properties;
different road types resulting in different physical and visual properties.

17. A land vehicle, being provided with a system according to claim 15 .

18. A road-terrain detection method for driver assistance systems, wherein the road-terrain detection method comprises
sensing the environment of a vehicle with at least one sensor;
transforming a sensor signal from the at least one sensor into at least one confidence map of local properties of the environment by using at least one base classifier;
generating, for base points on the at least one confidence map, each reflecting a specific location in the environment of the vehicle, spatial features for the local properties based on the at least one confidence map; and
classifying, based on the generated spatial features, locations in the environment of the vehicle to a certain category of road terrain corresponding to the type of surface of the road, wherein the method comprises
a base classifier for visual boundary classification, in order to find visual features that discriminate a road boundary from a road-like area, wherein the method
includes feature generation,
uses a positive training set with polygonal training regions for the road boundary, and
uses a negative training set with polygonal training regions for road-like area excluding areas with lane-markings.

19. The method according to claim 18 , wherein the visual features comprise curbstones.

20. The method according to claim 18 , wherein feature generation comprises generating at least one of color, texture, appearance, flow, or depth.

21. A non-transitory computer-readable medium encoded with instructions that, when run on a computing device in a land vehicle, perform a method, the method comprising:
sensing the environment of a vehicle with at least one sensor;
transforming a sensor signal from the at least one sensor into at least one confidence map of local properties of the environment by using at least one base classifier;
generating, for base points on the at least one confidence map, each reflecting a specific location in the environment of the vehicle, spatial features for the local properties based on the at least one confidence map; and
classifying, based on the generated spatial features, locations in the environment of the vehicle to a certain category of road terrain corresponding to a type of surface of a road, wherein the generation of a spatial feature for a specific location in the environment of the vehicle comprises
extracting a ray, wherein a ray is defined as a directed line with a certain a

Assignees

• Fritsch Jannik
• Kühnl Tobias
• Honda Res Inst Europe Gmbh

Inventors

• Fritsch Jannik
• Kühnl Tobias

Classifications

• G06T7/0087

  Physics · mapped topic

• G06T2207/30256

  Lane; Road marking · CPC title

• G01S15/89

  for mapping or imaging · CPC title

• G01S13/89 Primary

  for mapping or imaging · CPC title

• G01S17/93

  for anti-collision purposes · CPC title