Artificial intelligence apparatus and method for determining inattention of driver

Patent metadata

Field	Value
Publication number	US-2021124962-A1
Application number	US-201916700470-A
Country	US
Kind code	A1
Filing date	Dec 2, 2019
Priority date	Oct 29, 2019
Publication date	Apr 29, 2021
Grant date	—

Abstract

Disclosed herein an artificial intelligence apparatus for determining inattention of a driver including a vibration sensor or a gyro sensor configured to sense movement of a driver's seat of a vehicle, a camera configured to receive image data including a face of a driver, a communication modem configured to receive vehicle status information from an ECU (Electronic Control Unit) of the vehicle, and a processor configured to generate movement information of the driver's seat using vibration sensor information received from the vibration sensor or gyro sensor information received from the gyro sensor, generate driver status information corresponding to the driver from the received image data, determine whether the driver is in an inattention status based on the movement information of the driver's seat, the driver status information and the vehicle status information, and output an inattention alarm if the driver is in the inattention status.

First claim

1 . An artificial intelligence apparatus for determining inattention of a driver, comprising:
a vibration sensor or a gyro sensor configured to sense movement of a driver's seat of a vehicle; a camera configured to receive image data including a face of a driver; a communication modem configured to receive vehicle status information from an ECU (Electronic Control Unit) of the vehicle; and a processor configured to:
generate movement information of the driver's seat using vibration sensor information received from the vibration sensor or gyro sensor information received from the gyro sensor,
generate driver status information corresponding to the driver from the received image data,
determine whether the driver is in an inattention status based on the movement information of the driver's seat, the driver status information and the vehicle status information, and
output an inattention alarm when the driver is in the inattention status,
wherein the processor is further configured to:
select an inattention determination model to be used to determine inattention of the driver in consideration of a type of the vehicle and identification information of the driver, and
determine inattention of the driver using the selected inattention determination model,
wherein the inattention determination model includes an artificial neural network, and is learned using a machine learning algorithm or a deep learning algorithm.

2 . The artificial intelligence apparatus of claim 1 ,
wherein the processor is configured to generate, as the driver status information, at least one of identification information of the driver, a face direction of the driver, a gaze direction of the driver, a distance between eyelids of the driver or a distance between lips of the driver from the image data using a face recognition model, and wherein the face recognition model includes a convolutional neural network (CNN) learned using a deep learning algorithm.

3 . The artificial intelligence apparatus of claim 2 , wherein the vehicle status information includes at least one of a speed of the vehicle, a revolution per minute (RPM) of an engine, a transmission gear state, pedal pressure, a steering angle or a lighting state of a turn signal lamp.

4 - 11 . (canceled)

12 . The artificial intelligence apparatus of claim 1 , wherein the inattention determination model is learned using training data which includes an input feature vector including at least one of the face direction, the gaze direction, the distance between the eyelids, the distance between the lips, the movement information of the driver's seat or the vehicle status information, and a labeled inattention corresponding to the input feature vector.

13 . The artificial intelligence apparatus of claim 12 , further comprising a microphone,
wherein the processor is configured to:
receive speech feedback of the driver via the microphone after the inattention alarm is output,
when the speech feedback is negative feedback, generate training data for update based on the negative feedback,
when the speech feedback is positive feedback, generate training data for update based on the positive feedback, and
update the inattention determination model using the generated training data for update.

14 . A method of determining inattention of a driver, comprising:
sensing movement of a driver's seat of a vehicle using a vibration sensor or a gyro sensor; generating movement information of the driver's seat using vibration sensor information received from the vibration sensor or gyro sensor information received from the gyro sensor; receiving image data including a face of a driver from a camera; generating driver status information corresponding to the driver from the received image data; receiving vehicle status information from an ECU (Electronic Control Unit) of the vehicle; determining whether the driver is in an inattention status based on the movement information of the driver's seat, the driver status information and the vehicle status information; and outputting an inattention alarm when the driver is in the inattention status, wherein the determining whether the driver is in the inattention status comprises:
selecting an inattention determination model to be used to determine inattention of the driver in consideration of a type of the vehicle and identification information of the driver, and
determining inattention of the driver using the selected inattention determination model,
wherein the inattention determination model includes an artificial neural network, and is learned using a machine learning algorithm or a deep learning algorithm.

15 . A non-transitory recording medium stored thereon a computer program for controlling a processor to perform a method of determining inattention of a driver, the method comprising:
sensing movement of a driver's seat of a vehicle using a vibration sensor or a gyro sensor; generating movement information of the driver's seat using vibration sensor information received from the vibration sensor or gyro sensor information received from the gyro sensor; receiving image data including a face of a driver from a camera; generating driver status information corresponding to the driver from the received image data; receiving vehicle status information from an ECU (Electronic Control Unit) of the vehicle; determining whether the driver is in an inattention status based on the movement information of the driver's seat, the driver status information and the vehicle status information; and outputting an inattention alarm when the driver is in the inattention status, wherein the determining whether the driver is in the inattention status comprises:
selecting an inattention determination model to be used to determine inattention of the driver in consideration of a type of the vehicle and identification information of the driver, and
determining inattention of the driver using the selected inattention determination model,
wherein the inattention determination model includes an artificial neural network, and is learned using a machine learning algorithm or a deep learning algorithm.

Assignees

• Lg Electronics Inc

Inventors

• Kim Sungsik

Classifications

• G06V10/7784

  based on feedback from supervisors · CPC title

• G06V40/18

  Eye characteristics, e.g. of the iris · CPC title

• G06V40/171

  Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships · CPC title

• G06V40/161

  Detection; Localisation; Normalisation · CPC title

• G06V40/172

  Classification, e.g. identification · CPC title