Visual perception assistance system and visual-perception target object detection system

Patent metadata

Field	Value
Publication number	US-10717447-B2
Application number	US-201616062591-A
Country	US
Kind code	B2
Filing date	Dec 16, 2016
Priority date	Dec 17, 2015
Publication date	Jul 21, 2020
Grant date	Jul 21, 2020

Abstract

The present invention includes: a visual perception target determination unit for determining a visual perception target by using a saliency model for determining that an object to be visually perceived at a glance is a visual perception target, a surprise model for determining that an object behaving abnormally is a visual perception target, and a normative model for determining that an object to be visually perceived by a viewing action of a driver serving as a norm is a visual perception target; and a visual guidance unit for determining whether or not an overlooked visual perception to which a line-of-sight direction detected by a line-of-sight detection unit is not directed is present, and guiding a line-of-sight of a driver toward the overlooked visual perception target, when the overlooked visual perception target is present.

First claim

The invention claimed is:

1. A visual perception assistance system, comprising:
a surrounding condition acquisition unit for acquiring a surrounding condition of a moving body to be driven by a driver;
a visual perception target determination unit for determining a visual perception target being an object that the driver should look at in a surrounding condition acquired by the surrounding condition acquisition unit;
a line-of-sight direction detection unit for detecting a line-of-sight direction of the driver; and
a visual guidance unit for determining whether or not an overlooked visual perception target to which a line-of-sight direction detected by the line-of-sight direction detection unit is not directed is present among the visual perception target determined by the visual perception target determination unit, and guiding a line-of-sight of the driver toward the overlooked visual perception target, when the overlooked visual perception target is present, wherein
the visual perception target determination unit determines the visual perception target, based on three visual perception models prepared in advance, and
the three visual perception models include a saliency model for determining that an object to be visually perceived at a glance is a visual perception target, a surprise model for determining that an object behaving abnormally is a visual perception target, and a normative model for determining that an object to be visually perceived by a viewing action of a driver serving as a norm is a visual perception target.

2. The visual perception assistance system according to claim 1 , wherein
the moving body is constituted by a vehicle, and
the surrounding condition acquisition unit is constituted by a camera for photographing an area ahead of the vehicle.

3. The visual perception assistance system according to claim 1 , wherein
the visual guidance unit performs visual guidance by displaying an index at a position of the overlooked visual perception target.

4. The visual perception assistance system according to claim 1 , further comprising:
a driver state detection unit for detecting the driver state, wherein
the visual guidance unit changes a degree of intensity of visual guidance depending on a driver state to be detected by the driver state detection unit.

5. The visual perception assistance system according to claim 4 , wherein
the visual guidance unit changes the degree of intensity of visual guidance by changing a degree of conspicuousness of an index to be displayed at a position of the overlooked visual perception target.

6. The visual perception assistance system according to claim 4 , wherein
the driver state detection unit detects at least a state that driving load is large as the driver state, and
when the driver state detection unit detects that the driving load is large, the visual guidance unit emphasizes visual guidance, as compared with a case where the driver state detection unit detects that the driving load is small.

7. The visual perception assistance system according to claim 4 , wherein
the driver state detection unit detects at least a state that the driver is absent-minded, as the driver state, and
when the driver state detection unit detects that the driver is absent-minded, the visual guidance unit emphasizes visual guidance, as compared with a case where the driver state detection unit detects that the driver is not absent-minded.

8. The visual perception assistance system according to claim 1 , wherein
when a plurality of the overlooked visual perception targets are present, the visual guidance unit ranks the plurality of overlooked visual perception targets in a viewing order, and performs visual guidance of the plurality of overlooked visual perception targets by the ranking.

9. A visual perception target detection system, comprising:
a surrounding condition acquisition unit for acquiring a surrounding condition of an observer; and
a visual perception target determination unit for determining a visual perception target being an object that the observer should look at in a surrounding condition acquired by the surrounding condition acquisition unit, wherein
the visual perception target determination unit determines the visual perception target, based on three visual perception models prepared in advance, and
the three visual perception models include a saliency model for determining that an object to be visually perceived at a glance is a visual perception target, a surprise model for determining that an object behaving abnormally is a visual perception target, and a normative model for determining that an object to be visually perceived by a viewing action of an observer serving as a norm is a visual perception target.

10. The visual perception target detection system according to claim 9 , further comprising:
a display unit for displaying a visual perception target determined by the visual perception target determination unit.

Assignees

• Mazda Motor

Inventors

• Isa Tadashi
• Yoshida Masatoshi
• Veale Richard
• Takeda Yusaku
• Hara Toshihiro
• Iwase Koji
• Kishi Atsuhide
• Nishikawa Kazuo
• Nozawa Takahide

Classifications

• B60K35/235

  with means for detecting the driver's gaze direction or eye points · CPC title

• B60K35/265

  Voice · CPC title

• B60K35/285

  for improving awareness by directing driver's gaze direction or eye points · CPC title

• G06V20/597 Primary

  Recognising the driver's state or behaviour, e.g. attention or drowsiness · CPC title

• G06V40/193

  Preprocessing; Feature extraction · CPC title