Occupant monitoring device, occupant monitoring method, and occupant monitoring program

Patent metadata

Field	Value
Publication number	US-11048952-B2
Application number	US-201916580593-A
Country	US
Kind code	B2
Filing date	Sep 24, 2019
Priority date	Sep 27, 2018
Publication date	Jun 29, 2021
Grant date	Jun 29, 2021

Abstract

An occupant monitoring device includes: an acquisition unit that acquires a captured image obtained by imaging a region in which there is a probability that a face of an occupant is present in a vehicle; a determination unit that determines whether the captured image acquired by the acquisition unit corresponds to at least a first image not including a feature portion of the face of the occupant but including at least a part of a body of the occupant or a second image not including the body of the occupant; and a processing unit that monitors a state change of the face of the occupant based on the captured image, and switches whether or not an occupant parameter set for monitoring the state change of the face of the occupant is to be reset according to a determination result in the determination unit.

First claim

What is claimed is:

1. An occupant monitoring device comprising:
at least one hardware processor configured to implement:
an acquisition unit that acquires a captured image obtained by imaging a region in which there is a probability that a face of an occupant is present in a vehicle;
a determination unit that determines whether the captured image acquired by the acquisition unit corresponds to at least a first image not including a feature portion of the face of the occupant but including at least a part of a body of the occupant or a second image not including the body of the occupant; and
a processing unit that monitors a state change of the face of the occupant based on the captured image, and switches whether or not an occupant parameter set for monitoring the state change of the face of the occupant is to be reset according to a determination result in the determination unit, wherein
the determination unit determines whether the captured image corresponds to the first image, the second image, or a third image including the face of the occupant,
in a case where the determination unit determines that the captured image corresponds to the first image, the processing unit holds the occupant parameter without detecting face information regarding the face of the occupant based on the captured image,
in a case where the determination unit determines that the captured image corresponds to the second image, the processing unit resets the occupant parameter without detecting the face information based on the captured image, and
in a case where the determination unit determines that the captured image corresponds to the third image, the processing unit detects the face information based on the captured image, and updates the occupant parameter.

2. The occupant monitoring device according to claim 1 , wherein
the acquisition unit successively acquires the captured image a plurality of times,
in a case where the determination unit determines that the captured image corresponds to the third image, the processing unit detects the face information, updates the occupant parameter, and then executes tracking of the face information based on the captured image acquired next by the acquisition unit, and
in a case where the determination unit determines that the captured image corresponds to the first image or the second image, the processing unit holds or resets the occupant parameter without detecting the face information, and then switches whether or not the face information is to be detected and whether the occupant parameter is to be reset, held, or updated according to a determination result in the determination unit with respect to the captured image acquired next by the acquisition unit without executing tracking of the face information.

3. The occupant monitoring device according to claim 2 , wherein
the acquisition unit successively acquires the captured image a plurality of times, and
in a case where the determination unit determines that the captured image corresponds to the second image, and then the determination unit determines that the captured image corresponds to the third image for the first time, the processing unit identifies the occupant based on the captured image, and executes a service set in advance for each occupant based on an identification result.

4. The occupant monitoring device according to claim 2 , wherein
the occupant parameter includes a position of the center of an eyeball of the occupant in a three-dimensional model representing a structure including a three-dimensional shape of the face of the occupant.

5. The occupant monitoring device according to claim 2 , wherein
the determination unit determines whether the captured image acquired by the acquisition unit corresponds to at least the first image or the second image based on a learned model generated by learning a learning image including information similar to the captured image and whether the learning image corresponds to at least the first image or the second image through machine learning.

6. The occupant monitoring device according to claim 1 , wherein
the acquisition unit successively acquires the captured image a plurality of times, and
in a case where the determination unit determines that the captured image corresponds to the second image, and then the determination unit determines that the captured image corresponds to the third image for the first time, the processing unit identifies the occupant based on the captured image, and executes a service set in advance for each occupant based on an identification result.

7. The occupant monitoring device according to claim 6 , wherein
the occupant parameter includes a position of the center of an eyeball of the occupant in a three-dimensional model representing a structure including a three-dimensional shape of the face of the occupant.

8. The occupant monitoring device according to claim 6 , wherein
the determination unit determines whether the captured image acquired by the acquisition unit corresponds to at least the first image or the second image based on a learned model generated by learning a learning image including information similar to the captured image and whether the learning image corresponds to at least the first image or the second image through machine learning.

9. The occupant monitoring device according to claim 1 , wherein
the occupant parameter includes a position of the center of an eyeball of the occupant in a three-dimensional model representing a structure including a three-dimensional shape of the face of the occupant.

10. The occupant monitoring device according to claim 9 , wherein
the determination unit determines whether the captured image acquired by the acquisition unit corresponds to at least the first image or the second image based on a learned model generated by learning a learning image including information similar to the captured image and whether the learning image corresponds to at least the first image or the second image through machine learning.

11. The occupant monitoring device according to claim 1 , wherein
the determination unit determines whether the captured image acquired by the acquisition unit corresponds to at least the first image or the second image based on a learned model generated by learning a learning image including information similar to the captured image and whether the learning image corresponds to at least the first image or the second image through machine learning.

12. The occupant monitoring device according to claim 1 , wherein
the occupant parameter includes a position of the center of an eyeball of the occupant in a three-dimensional model representing a structure including a three-dimensional shape of the face of the occupant.

13. The occupant monitoring device according to claim 1 , wherein
the determination unit determines whether the captured image acquired by the acquisition unit corresponds to at least the first image or the second image based on a learned model generated by learning a learning image including information similar to the captured image and whether the learning image corresponds to at least the first image or the second image through machine learning.

14. An occupant monitoring method comprising:
an acquisition step of acquiring a captured image obtained by imaging a region in which there is a probability that a face of an occupant is present in a vehicle;
a determination step of determining whether the captured image acquired in the acquisition step corresponds to at least a first image not including a feature portion of the face of the occupant but including at least a part of a

Assignees

• Aisin Seiki

Inventors

• Osuga Shin
• Kato Takashi
• Yamada Yuya
• Hattori Tetsuya
• Yamada Yoshiyuki
• Kawai Yuta

Classifications

• G06V20/597

  Recognising the driver's state or behaviour, e.g. attention or drowsiness · CPC title

• G06T7/70 Primary

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V40/166

  using acquisition arrangements · CPC title

• G06V40/161

  Detection; Localisation; Normalisation · CPC title