Apparatus for determining visual confirmation target

Patent metadata

Field	Value
Publication number	US-10977506-B2
Application number	US-201916663532-A
Country	US
Kind code	B2
Filing date	Oct 25, 2019
Priority date	Oct 29, 2018
Publication date	Apr 13, 2021
Grant date	Apr 13, 2021

Abstract

An apparatus for determining a visual confirmation target, the apparatus includes a gaze detection portion detecting a gaze direction of a driver for a vehicle, a vehicle information acquisition portion, an image acquisition portion acquiring a captured image, a gaze region extraction portion extracting a gaze region of the driver within the captured image based on a detection result of the gaze direction, a candidate detection portion recognizing objects included in the captured image, generating a top-down saliency map based on the captured image and the vehicle information, and detecting an object having saliency in the top-down saliency map among the recognized objects as a candidate for a visual confirmation target, and a visual confirmation target determination portion determining a visual confirmation target on a basis of an extraction result of the gaze region and a detection result of the candidate for the visual confirmation target.

First claim

The invention claimed is:

1. An apparatus for determining a visual confirmation target, the apparatus comprising:
a gaze detection portion detecting a gaze direction of a driver for a vehicle;
a vehicle information acquisition portion acquiring vehicle information indicating a driving state of the vehicle;
an image acquisition portion acquiring a captured image from an imaging device that captures an image in surroundings of the vehicle;
a gaze region extraction portion extracting a gaze region at which the driver looks within the captured image based on a detection result of the gaze direction detected by the gaze detection portion;
a candidate detection portion recognizing objects included in the captured image, generating a top-down saliency map for the captured image based on the captured image and the vehicle information, and detecting an object having saliency in the top-down saliency map among the recognized objects as a candidate for a visual confirmation target at which the driver looks; and
a visual confirmation target determination portion determining a visual confirmation target at which the driver looks on a basis of an extraction result of the gaze region and a detection result of the candidate for the visual confirmation target.

2. The apparatus according to claim 1 , wherein the candidate detection portion generates the top-down saliency map based on the captured image acquired at a predetermined position among a plurality of the captured images acquired by the image acquisition portion,
the gaze detection portion detects a gaze direction candidate related to driving of the vehicle among candidates for every gaze direction of the driver based on the top-down saliency map.

3. The apparatus according to claim 1 , wherein the gaze detection portion detects a direction of the face of the driver,
the candidate detection portion calculates a degree of reliability that the driver looks at the candidate for the visual confirmation target based on the direction of the face of the driver detected by the gaze detection portion,
the visual confirmation target determination portion determines the visual confirmation target at which the driver looks on a basis of a distance between the gaze region within the captured image and the candidate for the visual confirmation target, and the degree of reliability of the candidate for the visual confirmation target.

4. The apparatus according to claim 2 , wherein the gaze detection portion detects a direction of the face of the driver,
the candidate detection portion calculates a degree of reliability that the driver looks at the candidate for the visual confirmation target based on the direction of the face of the driver detected by the gaze detection portion,
the visual confirmation target determination portion determines the visual confirmation target at which the driver looks on a basis of a distance between the gaze region within the captured image and the candidate for the visual confirmation target, and the degree of reliability of the candidate for the visual confirmation target.

5. The apparatus according to claim 1 , wherein the gaze region extraction portion extracts a region that is most likely to be looked by the driver among regions in the captured image obtained on a basis of the gaze direction detected by the gaze detection portion as the gaze region.

6. The apparatus according to claim 2 , wherein the gaze region extraction portion extracts a region that is most likely to be looked by the driver among regions in the captured image obtained on a basis of the gaze direction detected by the gaze detection portion as the gaze region.

7. The apparatus according to claim 3 , wherein the gaze region extraction portion extracts a region that is most likely to be looked by the driver among regions in the captured image obtained on a basis of the gaze direction detected by the gaze detection portion as the gaze region.

8. The apparatus according to claim 5 , wherein the gaze region extraction portion extracts a region that is most likely to be looked by the driver among regions in the captured image obtained on a basis of the gaze direction detected by the gaze detection portion as the gaze region.

Assignees

• Aisin Seiki

Inventors

• Kogure Shunsuke
• Osuga Shin
• Murakami Shinichiro

Classifications

• G06V40/18

  Eye characteristics, e.g. of the iris · CPC title

• G06V20/59

  inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions · CPC title

• G06V20/597

  Recognising the driver's state or behaviour, e.g. attention or drowsiness · CPC title

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title