Pedestrian behavior predicting device and pedestrian behavior predicting method

Patent metadata

Field	Value
Publication number	US-9904846-B2
Application number	US-201114346015-A
Country	US
Kind code	B2
Filing date	Sep 20, 2011
Priority date	Sep 20, 2011
Publication date	Feb 27, 2018
Grant date	Feb 27, 2018

Abstract

According to the present invention, a pedestrian is detected from an imaged image and a partial image including the pedestrian is extracted, shape information of the pedestrian acquired from the extracted partial image is accumulated and the shape information of a predetermined time before and the current shape information are compared using the accumulated shape information to detect change in the movement of the pedestrian, discontinuous movement estimating information indicating a discontinuous movement of the pedestrian that occurs following the change in the movement of the pedestrian is acquired from a storage means at the time the change in the movement of the pedestrian is detected, and a behavior of the pedestrian is predicted using the acquired discontinuous movement estimating information.

First claim

The invention claimed is:

1. A pedestrian behavior predicting device comprising:
a processor including hardware; and
a storage unit including memory storing instructions executable by the processor and for storing shape information of a pedestrian, wherein the processor executes the instructions stored in the memory to cause the pedestrian behavior predicting device to:
detect a pedestrian from an imaged image; acquired at a current time
extract a partial image including the detected pedestrian from the imaged image;
acquire shape information of the pedestrian from the extracted partial image, the shape information of the pedestrian including at least one of a luminance of the partial image, an edge of the partial image, and a color of the partial image;
accumulate the acquired shape information of the pedestrian in the storage unit;
detect a change in movement of the pedestrian by comparing shape information of the pedestrian from an image captured a predetermined time before the current time and the acquired shape information of the pedestrian using the accumulated shape information of the pedestrian,
store first images and pieces of discontinuous movement estimating information in the storage unit such that each first image is associated with each piece of discontinuous movement estimating information, the first image being an image that was acquired at a time when a first change in the movement of the pedestrian is detected, the piece of discontinuous movement estimating information being information indicating a discontinuous movement of the pedestrian that occurs following the first change in the movement of the pedestrian;
wherein when a second change in the movement of the pedestrian is detected, search, from the first images stored in the storage unit, a first image corresponding to a second image when the second change in the movement of the pedestrian is detected;
acquire a piece of discontinuous movement estimating information associated with the searched first image; and
predict a behavior of the pedestrian using the acquired piece of discontinuous movement estimating information.

2. The pedestrian behavior predicting device according to claim 1 , wherein
the processor further executes the instructions stored in the memory to cause the pedestrian behavior predicting device to:
detect the change in the movement of the pedestrian by acquiring a feature amount distribution representing the shape information of the pedestrian acquired from the extracted partial image, normalize the shape information represented by the acquired feature amount distribution, accumulate the normalized shape information, and compare the shape information of the image captured a predetermined time before the current time and the acquired shape information using the accumulated shape information.

3. The pedestrian behavior predicting device according to claim 2 , wherein
the processor further executes the instructions stored in the memory to cause the pedestrian behavior predicting device to:
calculate an optical flow of the imaged image and acquire the discontinuous movement estimating information using the optical flow at the time the first change in the movement of the pedestrian is detected.

4. The pedestrian behavior predicting device according to claim 3 , wherein
the discontinuous movement estimating information includes at least one of a moving direction and a moving speed of the pedestrian.

5. The pedestrian behavior predicting device according to claim 2 , wherein
the processor further executes the instructions stored in the memory to cause the pedestrian behavior predicting device to:
acquire a position of the pedestrian on the imaged image,
generate continuous movement estimating information indicating a continuous movement of the pedestrian that occurs following movement of the position of the pedestrian based on a history of positions of the pedestrian acquired, and
predict the behavior of the pedestrian based on the continuous movement estimating information generated.

6. The pedestrian behavior predicting device according to claim 2 , wherein
the discontinuous movement estimating information includes at least one of a moving direction and a moving speed of the pedestrian.

7. The pedestrian behavior predicting device according to claim 2 , wherein
the processor further executes the instructions stored in the memory to cause the pedestrian behavior predicting device to:
acquire the feature amount distribution of the pedestrian from the extracted partial image using a predetermined feature amount,
normalize the acquired feature amount distribution and acquire a probability distribution corresponding to the feature amount distribution,
accumulate the acquired probability distribution, and
calculate a difference between the probability distribution of an image captured a predetermined time before the current time and the acquired probability distribution using a predetermined scale, and detect the change in the movement of the pedestrian when the calculated difference is greater than a predetermined threshold value.

8. The pedestrian behavior predicting device according to claim 1 , wherein
the processor further executes the instructions stored in the memory to cause the pedestrian behavior predicting device to:
calculate an optical flow of the imaged image and acquire the discontinuous movement estimating information using the optical flow at the time the first change in the movement of the pedestrian is detected.

9. The pedestrian behavior predicting device according to claim 8 , wherein
the processor further executes the instructions stored in the memory to cause the pedestrian behavior predicting device to:
acquire a position of the pedestrian on the imaged image,
generate continuous movement estimating information indicating a continuous movement of the pedestrian that occurs following movement of the position of the pedestrian based on a history of positions of the pedestrian acquired, and
predict the behavior of the pedestrian based on the continuous movement estimating information generated.

10. The pedestrian behavior predicting device according to claim 8 , wherein
the discontinuous movement estimating information includes at least one of a moving direction and a moving speed of the pedestrian.

11. The pedestrian behavior predicting device according to claim 1 , wherein
the processor further executes the instructions stored in the memory to cause the pedestrian behavior predicting device to:
acquire a position of the pedestrian on the imaged image,
generate continuous movement estimating information indicating a continuous movement of the pedestrian that occurs following movement of the position of the pedestrian based on a history of positions of the pedestrian acquired, and
predict the behavior of the pedestrian based on the continuous movement estimating information generated.

12. The pedestrian behavior predicting device according to claim 1 , wherein
the discontinuous movement estimating information includes at least one of a moving direction and a moving speed of the pedestrian.

13. The pedestrian behavior predicting device according to claim 1 , wherein
the processor further executes the instructions stored in the memory to cause the pedestrian behavior predicting device to:
acquire a feature amount distribution of the pedestrian from the extracted partial image using a predetermined feature amount,
normalize the acquired feature amount distribution and acquire a probability distribution corresponding to the feature amount distribution,
accumulate the acquired probability distribu

Assignees

• Kindo Toshiki
• Ogawa Masahiro
• Funayama Ryuji
• Toyota Motor Co Ltd

Inventors

• Kindo Toshiki
• Ogawa Masahiro
• Funayama Ryuji

Classifications

• B60W2554/00

  Input parameters relating to objects · CPC title

• B60W2420/40

  Photo, light or radio wave sensitive means, e.g. infrared sensors · CPC title

• G08G1/08

  according to detected number or speed of vehicles · CPC title

• G08G1/166 Primary

  for active traffic, e.g. moving vehicles, pedestrians, bikes · CPC title

• G06K9/00369 Primary

  Physics · mapped topic