Occupant monitoring systems and methods

Patent metadata

Field	Value
Publication number	US-10787189-B2
Application number	US-201816055827-A
Country	US
Kind code	B2
Filing date	Aug 6, 2018
Priority date	Dec 30, 2014
Publication date	Sep 29, 2020
Grant date	Sep 29, 2020

Abstract

Various implementations relate to an operator monitoring system (OMS). Certain implementations include an OMS coupled to a rotatable portion of a steering wheel assembly of a vehicle. For example, the OMS may include an imaging unit, such as a camera, that is coupled to a central hub portion of the steering wheel assembly. The imaging unit has a field of view directed toward one or more occupants in the vehicle and is configured to capture an image signal corresponding to an imaging area in the field of view. The imaging area can be configured to encapsulate an expected position of one or more vehicle occupants. The OMS also includes one or more processing units in electrical communication with the imaging unit that receives and processes the image signal from the imaging unit to determine an occupant state and, in some implementations, provide feedback based on the determined occupant state.

First claim

What is claimed is:

1. An occupant monitoring system (OMS) for monitoring at least one occupant in a vehicle, the vehicle having a steering wheel assembly configured to rotate relative to a steering column, the OMS comprising:
an imaging unit coupled to a steering wheel assembly having a field of view inside of the vehicle configured to capture an image signal corresponding to an imaging area in the vehicle;
a processing unit including a processor and a memory operably coupled to the processor, the memory having computer-executable instructions stored thereon that, when executed by the processor, cause the processor to:
receive the image signal from the imaging unit;
analyze the image signal to determine an occupant state; and
output a signal based on the determined occupant state, where the output signal is outputted to another vehicle system to provide information about the occupant state to the other vehicle system, wherein the output signal includes instructions for at least one of the OMS and the other vehicle system to adjust a vehicle parameter based on the occupant state, and an adjusted vehicle parameter is used to warn the occupant;
wherein the adjusted vehicle parameter includes at least one of operation of a light source, operation of a cabin audio system, sensitivity of a steering wheel of the steering wheel assembly to user input, sensitivity of a braking system to user input, sensitivity of a vehicle accelerator to user input, operation of an interior light, a temperature of a vehicle cabin, a scent provided to an occupant cabin of the vehicle, output of a vehicle communication system, an operation of an autonomous driving state of the vehicle, and a tactile feedback to the occupant via a haptic feedback device; and
wherein the tactile feedback is provided to the steering wheel assembly and includes at least one of vibration, heat, and cooling effect.

2. The OMS of claim 1 , wherein the light source includes at least one of an imaging unit light source, a light bar system disposed on another portion of the steering wheel assembly, and a cabin light source,
wherein the adjusted vehicle parameter includes adjusting at least one of a wavelength and an intensity of light being emitted by corresponding one of the imaging unit light source, light bar system, and cabin light source.

3. The OMS of claim 1 , wherein the haptic feedback device includes at least one of a heater pad disposed around a rim of the steering wheel assembly and a vibratory exciter disposed within the rim or hub of the steering wheel assembly.

4. The OMS of claim 1 , wherein the imaging unit includes a camera.

5. The OMS of claim 1 , further including a light source configured to illuminate the imaging area, wherein the light source comprises an infrared light source.

6. An occupant monitoring system (OMS) for monitoring at least one occupant in a vehicle, the vehicle having a steering wheel assembly configured to rotate relative to a steering column, the OMS comprising:
an OMS imaging unit coupled to the steering wheel assembly having a field of view inside of the vehicle configured to capture an image signal corresponding to an imaging area in the vehicle;
an OMS processing unit associated with the imaging unit, the OMS processing unit including an OMS processor and an OMS memory operably coupled to the OMS processor, the OMS memory having computer-executable instructions stored thereon;
a vehicle processing unit associated with another vehicle system, the vehicle processing unit including a vehicle processor and a vehicle memory operably coupled to the vehicle processor, the vehicle memory having computer-executable instructions stored thereon;
wherein the computer-executable instructions of the OMS processing unit and the computer-executable instructions of the vehicle processing unit, that, when executed by their corresponding processor, cause at least one of the corresponding processors to:
capture an image of the occupant;
receive the image signal from the imaging unit corresponding to the captured image;
analyze the image signal to determine an occupant state; and
output a signal based on the determined occupant state, where the output signal is outputted to at least one of the OMS and the other vehicle system to provide information about the occupant state to the corresponding OMS and other vehicle system,
wherein the output signal includes instructions for at least one of the OMS and the other vehicle system to change a vehicle parameter,

wherein at least one of the OMS processing unit and the vehicle processing unit are configured to store feature information related to the image signal captured just prior to the vehicle being turned off, the stored feature information stored in a temporary memory,
wherein the computer-executable instructions of the OMS processing unit and the vehicle processing unit, further cause at least one of the corresponding processors to retrieve the stored feature information upon vehicle start up and compare the stored feature information with an initial image of the occupant captured at vehicle start up to identify the occupant.

7. An occupant monitoring system (OMS) for monitoring at least one occupant in a vehicle, the vehicle having a steering wheel assembly configured to rotate relative to a steering column, the OMS comprising:
an imaging unit coupled to a steering wheel assembly having a field of view inside of the vehicle configured to capture an image signal corresponding to an imaging area in the vehicle;
a processing unit including a processor and a memory operably coupled to the processor, the memory having computer-executable instructions stored thereon that, when executed by the processor, cause the processor to:
receive the image signal from the imaging unit;
analyze the image signal to determine an occupant state;

output a signal based on the determined occupant state, where the output signal is outputted to another vehicle system to provide information about the occupant state to the other vehicle system,
wherein determining that the occupant is distracted includes monitoring the occupant's mouth movements to determine that the occupant is vocalizing and comparing the mouth movements with an audio signal captured by the vehicle and using the comparison to identify and filter out speech from another occupant and/or background noise,
wherein analyzing the image signal to determine an occupant state includes:
processing the image signal to identify an analysis region;
analyzing the analysis region to determine at least one occupant information parameter;
analyzing the at least one occupant information parameter to determine the occupant state;
wherein analyzing the occupant information parameter to determine the occupant state includes determining at least one of:
the occupant's eyes are continuously open or closed;
the occupant is looking outside of the vehicle in a direction of travel;
the occupant is looking within the vehicle;
the occupant is distracted;
a relative alertness of the occupant;
an emotional state of the occupant;
a position or orientation of the occupant relative to a predetermined position, and
a physiological condition of the occupant.

8. The OMS of claim 7 , wherein determining the physiological condition comprises monitoring movement of the occupant's nose and/or mouth to determine a respiration rate of the occupant,
wherein a medical emergency is determined when the determined respiration rate less than a minimum threshold respiration rate and/or greater than a maximum threshold respiration rate.

9. The OMS of claim 7 , wherein identifying the analysis region comprises determining a position and/or orientation of at least one of a to

Assignees

• Joyson Safety Systems Acquisition Llc

Inventors

• Lisseman Jason Carl
• Cech Len
• Gardner Valerie

Classifications

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• H04N23/51

  Housings · CPC title

• H04N23/20

  for generating image signals from infrared radiation only · CPC title

• G06V20/593

  Recognising seat occupancy · CPC title

• G06V20/59

  inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions · CPC title