Method and system for proactively recognizing an action of a road user

Patent metadata

Field	Value
Publication number	US-9460343-B2
Application number	US-201514696903-A
Country	US
Kind code	B2
Filing date	Apr 27, 2015
Priority date	Apr 25, 2014
Publication date	Oct 4, 2016
Grant date	Oct 4, 2016

Abstract

A method and system are provided to proactively recognize an action of a road user in road traffic, wherein an image of the road user, which is structured in a pixel-wise manner, is captured by way of at least one camera, and corresponding image data is generated. Image data of multiple pixels is grouped in each case by cells, wherein the image comprises multiple cells. A respective centroid is determined based on the image data within a cell. For each of the pixels, the respective distance from the centroids of a plurality of cells is ascertained, wherein a feature vector that is assigned to the pixel is formed based on coordinates of the respective pixel and the centroids. The feature vector is compared to at least one reference vector cluster, and a pose is associated with the road user based on the comparison.

First claim

What is claimed is:

1. A method for proactively recognizing an action of a road user in road traffic, the method comprising the acts of:
capturing an image of the road user, which is structured in a pixel-wise manner, by way of at least one camera, and generating corresponding image data;
grouping pixels from the image data into a corresponding plurality of cells, wherein the image comprises the plurality of cells;
determining a respective cell centroid based on the image data for each of the plurality of cells;
for each of the pixels, determining a distance from the cell centroids of the plurality of cells, wherein a feature vector is assigned to each pixel based on coordinates of the respective pixel and the cell centroids; and
comparing the feature vector for each of the pixels to at least one reference vector cluster;
associating a pose, from a predetermined number of poses, with the road user based on said comparing; and
determining an expected action of the road user based at least in part on the associated pose.

2. The method according to claim 1 , wherein for determining the respective cell centroid, the method further comprises the acts of:
filtering the image data captured by the camera to the effect that data of a contiguous image area is created, wherein the road user is depicted in the image area and at least some of the other image components captured by the camera are filtered out; and
wherein the plurality of cells comprise the contiguous image area, and further wherein each of the plurality of cells comprises at least one pixel of the contiguous area.

3. The method according to claim 2 , wherein a respective pixel-wise orientation of the road user relative to the camera is associated in the course of the comparison of the respective feature vectors of the respective pixels to the respective at least one reference vector cluster and, based on the associated respective pixel-wise orientation, an orientation of the road user is associated.

4. The method according to claim 3 , wherein, based on the associated orientation, the feature vectors are compared pixel-wise to the respective reference vector clusters thereof and/or further reference vector clusters, which are classified according to possible poses of the road user, and in particular according to the associated orientation, and the pose is associated by way of the result of the comparison.

5. The method according to claim 1 , wherein the comparison of the feature vectors to the reference vector clusters is carried out by way of a random forest method, wherein decision trees constructed during a learning process are used, which comprise the reference vector clusters.

6. The method according to claim 1 , wherein further data is detected and used to proactively recognize the action of the road user, such further data comprising surroundings data, geographic data, measurement data regarding the road user and/or data of the motor vehicle.

7. The method according to claim 1 , wherein a stereo camera system is provided, which comprises first and second cameras, wherein the image of the road user is captured three-dimensionally using image data.

8. A system for proactively recognizing an action of a road user in road traffic, comprising:
at least one camera, which is usable to capture an image of the road user that is structured in a pixel-wise manner, and which generates corresponding image data; and
a data processing system configured to:
group pixels from the image data into a corresponding plurality of cells, wherein the image comprises the plurality of cells;
determine a respective cell centroid based on the image data for each of the plurality of cells;
ascertain, for each of the pixels, the respective distance from the cell centroids of the plurality of cells, wherein a feature vector is assigned to each pixel based on coordinates of the respective pixel and the cell centroids; and
compare the feature vector for each of the pixels to at least one reference vector cluster;
associate a pose, from a predetermined number of poses, with the road user based on the comparison; and
determine an expected action of the road user based at least in part on the associated pose.

9. The method of claim 1 , further comprising issuing a warning to a vehicle driver in response to the determined expected action of the road user.

10. The method of claim 1 , further comprising associating poses, from the predetermined number of poses, with the road user for each of the pixels comprising the image data,
wherein the method further comprises determining a final pose based on the associated poses for each of the pixels, and
wherein determining the expected action comprises determining the expected action of the road user based at least in part on the final pose.

11. The system of claim 8 , wherein the data processing system is further configured to issue a warning to a vehicle driver in response to the determined expected action of the road user.

12. The system of claim 8 , wherein the data processing system is further configured to:
associate poses, from the predetermined number of poses, with the road user for each of the pixels comprising the image data,
determine a final pose based on the associated poses for each of the pixels, and
determine the expected action based at least in part on the final pose.

Assignees

• Bayerische Motoren Werke Ag

Inventors

• Suerth Daniel
• Szczot Magdalena
• Lallemand Joe

Classifications

• G06V40/11

  Hand-related biometrics; Hand pose recognition · CPC title

• G06V20/10 Primary

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title

• G06V10/42

  Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation · CPC title

• G06T2207/30261

  Obstacle · CPC title

• G06T2207/30196

  Human being; Person · CPC title