Driver distraction detection system

Patent metadata

Field	Value
Publication number	US-2016267335-A1
Application number	US-201514657070-A
Country	US
Kind code	A1
Filing date	Mar 13, 2015
Priority date	Mar 13, 2015
Publication date	Sep 15, 2016
Grant date	—

Abstract

Embodiments are described for determining and responding to driver distractions. An example in-vehicle computing system of a vehicle includes an external device interface communicatively connecting the in-vehicle computing system to a mobile device, an inter-vehicle system communication module communicatively connecting the in-vehicle computing system to one or more vehicle systems of the vehicle, a processor, and a storage device storing instructions executable by the processor to receive image data from the mobile device, and determine a driver state based on the received image data. The instructions are further executable to receive vehicle data from one or more of the vehicle systems, determine a vehicle state based on the vehicle data, determine a distraction severity level based on the driver state and the vehicle state, and control one or more devices of the vehicle to perform a selected action based on the distraction severity level.

First claim

1 . An in-vehicle computing system of a vehicle, the in-vehicle computing system comprising:
an external device interface communicatively connecting the in-vehicle computing system to a mobile device; an inter-vehicle system communication module communicatively connecting the in-vehicle computing system to one or more vehicle systems of the vehicle; a processor; and a storage device storing instructions executable by the processor to:
receive image data from the mobile device via the external device interface, the image data imaging a driver and a driver environment;
determine a driver state based on the received image data;
responsive to determining that the driver state indicates that the driver is distracted:
receive vehicle data from one or more of the vehicle systems via the inter-vehicle system communication module;
determine a vehicle state based on the vehicle data;
determine a distraction severity level based on the driver state and the vehicle state; and
control one or more devices of the vehicle to perform a selected action based on the distraction severity level.

2 . The in-vehicle computing system of claim 1 , wherein
performing the selected action comprises presenting a visual warning responsive to the distraction severity level being within a first range; performing the selected action comprises presenting an audio warning responsive to the distraction severity level being within a second range; and performing the selected action comprises performing an automatic adjustment of a vehicle control warning responsive to the distraction severity level being within a third range.

3 . The in-vehicle computing system of claim 2 , further comprising a display device, wherein the visual warning comprises visual alert presented via the display device.

4 . The in-vehicle computing system of claim 2 , wherein the audio warning comprises an audio alert presented via one or more speakers in the vehicle.

5 . The in-vehicle computing system of claim 2 , wherein automatic adjustment of the vehicle control comprises automatic adjustment of engine operation.

6 . The in-vehicle computing system of claim 1 , wherein the mobile device comprises a wearable device including at least an outward-facing camera having a field of view that includes a vehicle environment, and a user-facing camera having a field of view that includes the driver of the vehicle.

7 . The in-vehicle computing system of claim 6 , wherein the instructions are further executable to receive position and motion data from the head-mounted device, determine the driver state based on image data and the position and motion data, and transmit image data comprising video data including one or more of the driver as imaged from the user-facing camera and the vehicle environment as imaged from the outward-facing camera.

8 . The in-vehicle computing system of claim 7 , wherein the image data includes an indication of driver gaze and objects of interest in a travel path of the vehicle, and wherein the driver state indicates that the driver is distracted responsive to determining that the driver gaze is directed to one or more objects of interest for a threshold period of time.

9 . The in-vehicle computing system of claim 8 , wherein the image data includes an indication of a trajectory of the objects of interest, and wherein the distraction severity level is based on a comparison of the trajectory of the objects of interest and the travel path of the vehicle.

10 . A method for an in-vehicle computing system of a vehicle, the method comprising:
receiving driver data from a wearable device, the driver data including image data from a driver-facing camera; receiving object data from one or more imaging devices of at least one of the wearable device and the vehicle, the object data including image data of a vehicle environment; receiving vehicle data from one or more vehicle systems, the vehicle data including an indication of an operating condition of the vehicle; determining whether a driver is distracted by correlating the driver data with the object data; responsive to determining that the driver is distracted, selecting an action based on correlating the driver data with the object data and the vehicle data and performing the selected action; and responsive to determining that the driver is not distracted, maintaining current operating parameters.

11 . The method of claim 10 , wherein the object data includes:
a total number of objects in a vehicle environment as determined from the image data from the one or more outward-facing cameras, and object trajectory information for each of the objects in the vehicle environment as determined from a comparison of a plurality of frames of image data from the one or more outward-facing cameras, the object trajectory information indicating an estimated trajectory of each of the objects.

12 . The method of claim 11 , wherein vehicle data includes vehicle trajectory information determined from one or more of a navigational system of the vehicle, sensor output of the wearable device, and image data from the one or more outward-facing cameras, the vehicle trajectory information indicating an estimated trajectory of the vehicle.

13 . The method of claim 12 , further comprising comparing the estimated trajectory of each of the objects and the estimated trajectory of the vehicle to determine intersections between the estimated trajectories of the objects and the estimated trajectory of the vehicle, wherein the selected action is selected based on the number of intersections between the estimated trajectories of the objects and the estimated trajectory of the vehicle.

14 . The method of claim 13 , wherein the selected action is further selected based on a vehicle speed and a gaze direction of the driver.

15 . The method of claim 13 , wherein a first action is selected responsive to determining that the estimated trajectory of at least one of the objects intersects the estimated trajectory of the vehicle and the vehicle speed is below a speed threshold, and a second action is selected responsive to determining that the estimated trajectory of at least one of the objects intersects the estimated trajectory of the vehicle, the vehicle speed is above the speed threshold, and the gaze direction of the driver intersected the current location of each of the at least one objects within a threshold time period and for a threshold duration.

16 . The method of claim 15 , wherein a third action is selected responsive to determining that the estimated trajectory of at least one of the objects intersects the estimated trajectory of the vehicle, the vehicle speed is above the speed threshold, and the gaze direction of the driver did not intersect the current location of each of the at least one objects within the threshold time period or for the threshold duration.

17 . The method of claim 16 , wherein the first action is a visual alert presented via a display in the vehicle, the second action is an audible alert presented via one or more speakers in the vehicle, and the third action is a vehicle control command issued from the in-vehicle computing system to a vehicle system to control engine operation of the vehicle.

18 . The method of claim 10 , wherein maintaining the current operating parameters comprises not performing an action that is based on correlating the driver data with the object data and the vehicle data.

19 . A system for identifying driver distraction, the system comprising:
a wearable device including a driver-f

Assignees

• Harman Int Ind

Inventors

• Hampiholi Vallabha Vasant

Classifications

• B60W40/08 Primary

  related to drivers or passengers · CPC title

• G02B2027/0138

  comprising image capture systems, e.g. camera · CPC title

• B60W2040/0827

  due to sleepiness · CPC title

• B60W2050/143

  Alarm means · CPC title

• B60W50/14

  Means for informing the driver, warning the driver or prompting a driver intervention · CPC title