Occupant behavior determining apparatus, occupant behavior determining method, and storage medium

Patent metadata

Field	Value
Publication number	US-11565707-B2
Application number	US-202117190940-A
Country	US
Kind code	B2
Filing date	Mar 3, 2021
Priority date	Mar 5, 2020
Publication date	Jan 31, 2023
Grant date	Jan 31, 2023

Abstract

An occupant behavior determining apparatus includes a surveillance camera that captures an image of an occupant of a vehicle to acquire an image; a face recognizing section that recognizes a face of the occupant based on the image; a posture recognizing section that recognizes a posture of the occupant based on the image; and a behavior determining section that determines a behavior of the occupant in a vehicle cabin based on a recognition result of the face recognizing section and a recognition result of the posture recognizing section.

First claim

What is claimed is:

1. An occupant behavior determining apparatus comprising:
a camera configured to capture an image of an occupant of a vehicle to acquire an image; and
one or more processors that execute computer-executable instructions stored in a memory, wherein
the one or more processors execute the computer-executable instructions to cause the occupant behavior determining apparatus to:
generate, from the image, a high-resolution image having a relatively high resolution compared to a low-resolution image, and the low-resolution image having a relatively low resolution compared to the high-resolution image;
determine presence or absence of a nearby object that is close to the camera, based on the low-resolution image;
in case of the presence of the nearby object, determine a behavior of the occupant in a vehicle cabin based on the low-resolution image; and
in case of the absence of the nearby object, recognize the face of the occupant and the posture of the occupant based on the high-resolution image, and
determine a behavior of the occupant in the vehicle cabin based on a recognition result of the face of the occupant and a recognition result of the posture of the occupant.

2. The occupant behavior determining apparatus according to claim 1 , wherein the one or more processors cause the occupant behavior determining apparatus to:
recognize an object held by the occupant based on the image; and
determine the behavior of the occupant based on the recognition result of the face of the occupant, the recognition result of the posture of the occupant, and a recognition result of the object held by the occupant.

3. The occupant behavior determining apparatus according to claim 1 , wherein
the camera is provided near a manipulated device of the vehicle that is to be manipulated by a finger of the occupant, and
the one or more processors cause the occupant behavior determining apparatus to determine the behavior of the occupant to be manipulation of the manipulated device if the nearby object is recognized.

4. An occupant behavior determining method, comprising:
generating, by a computer, from an image of the occupant captured by a camera, a high-resolution image having a relatively high resolution compared to a low-resolution image, and the low-resolution image having a relatively low resolution compared to the high-resolution image;
determining, by the computer, presence or absence of a nearby object that is close to the camera, based on the low-resolution image;
in case of the presence of the nearby object, determining, by the computer, a behavior of the occupant in a vehicle cabin based on the low-resolution image; and
in case of the absence of the nearby object, recognizing, by the computer, the face of the occupant and the posture of the occupant based on the high-resolution image, and
determining, by the computer, a behavior of the occupant in the vehicle cabin based on a recognition result of the face of the occupant and a recognition result of the posture of the occupant.

5. The occupant behavior determining method according to claim 4 , further comprising:
recognizing, by the computer, an object held by the occupant based on the image; and
determining, by the computer, the behavior of the occupant based on the recognition result of the face of the occupant, the recognition result of the posture of the occupant, and a recognition result of the object held by the occupant.

6. The occupant behavior determining method according to claim 4 ,
wherein
the camera is provided near a manipulated device of the vehicle that is to be manipulated by a finger of the occupant, and
the occupant behavior determining method further comprises determining, by the computer, the behavior of the occupant to be manipulation of the manipulated device if the nearby object is recognized.

7. A non-transitory computer-readable storage medium storing a program for causing a computer to:
generate, from an image of the occupant captured by a camera, a high-resolution image having a relatively high resolution compared to a low-resolution image, and the low-resolution image having a relatively low resolution compared to the high-resolution image;
determine presence or absence of a nearby object that is close to the camera, based on the low-resolution image;
in case of the presence of the nearby object, determine a behavior of the occupant in a vehicle cabin based on the low-resolution image; and
in case of the absence of the nearby object, recognize the face of the occupant and the posture of the occupant based on the high-resolution image, and
determine a behavior of the occupant in the vehicle cabin based on a recognition result of the face of the occupant and a recognition result of the posture of the occupant.

8. The non-transitory computer-readable storage medium according to claim 7 , wherein the program further causes the computer to:
recognize an object held by the occupant based on the image; and
determine the behavior of the occupant based on the recognition result of the face of the occupant, the recognition result of the posture of the occupant, and a recognition result of the object held by the occupant.

Assignees

• Honda Motor Co Ltd

Inventors

• Kusaka Takashi
• Kagiya Yuki

Classifications

• G06V20/20

  in augmented reality scenes · CPC title

• B60W40/09 Primary

  Driving style or behaviour · CPC title

• B60W2540/225

  Direction of gaze · CPC title

• B60W2540/229

  Attention level, e.g. attentive to driving, reading or sleeping · CPC title

• G06V20/597 Primary

  Recognising the driver's state or behaviour, e.g. attention or drowsiness · CPC title