Predicting vehicle movements based on driver body language

Patent metadata

Field	Value
Publication number	US-10423847-B2
Application number	US-201715714594-A
Country	US
Kind code	B2
Filing date	Sep 25, 2017
Priority date	Nov 4, 2015
Publication date	Sep 24, 2019
Grant date	Sep 24, 2019

Abstract

Systems, methods, and devices for predicting driver intent and future movements of a human driven vehicles are disclosed herein. A computer implemented method includes receiving an image of a proximal vehicle in a region near a vehicle. The method includes determining a region of the image that contains a driver of the proximal vehicle, wherein determining the region comprises determining based on a location of one or more windows of the proximal vehicle. The method includes processing image data only in the region of the image that contains the driver of the proximal vehicle to detect a driver's body language.

First claim

What is claimed is:

1. A computer implemented method comprising:
determining a region of an image that contains a driver of a proximal vehicle based on a location of one or more windows of the proximal vehicle;
processing image data only in the region of the image that contains the driver of the proximal vehicle to detect a driver's body language; and
determining a vehicle maneuver for the vehicle based on the driver's body language.

2. The computer implemented method of claim 1 , wherein detecting the driver's body language comprises detecting one or more of a head orientation, a gaze direction, and a gesture of the driver.

3. The computer implemented method of claim 2 , further comprising accessing a database or model that correlates one or more of the head orientation, the gaze direction, and the gesture with one or more future vehicle movements for the proximal vehicle.

4. The computer implemented method of claim 3 , wherein determining the vehicle maneuver based on the driver's body language comprises determining the vehicle maneuver based on one or more future vehicle movements of the proximal vehicle correlated with the driver's body language.

5. The computer implemented method of claim 1 , further comprising locating the proximal vehicle within the image.

6. The computer implemented method of claim 1 , wherein determining the region of the image that contains the driver comprises determining the region based on a predicted location of a driver's seat in the vehicle.

7. The computer implemented method of claim 1 , wherein a database or model correlates one or more of the following with the one or more future vehicle movements:
a waving motion of a hand;
a hand gesture comprising a palm facing toward the vehicle with fingers upward;
a gaze direction of the driver for a threshold period of time;
a series of head movements; and
a series of changes in gaze direction.

8. A system comprising:
one or more processors;
computer readable media storing instructions that, when executed by the one or more processors, cause the system to:
capture an image of a proximal vehicle;
identify a sub-portion of the image corresponding to an area where a driver of the proximal vehicle is located, wherein the boundary component identifies the sub-portion by identifying one or more windows of the proximal vehicle;
detect a driver's body language by processing data within only the sub-portion of the image; and
predict future motion of the proximal vehicle based on the driver's body language detected by the body language component.

9. The system of claim 8 , wherein the instructions cause the system to detect a driver's body language by identifying one or more of a driver's head orientation, a gaze direction, and a gesture.

10. The system of claim 8 , wherein the instructions cause the system to locate the proximal vehicle within the image.

11. The system of claim 8 , wherein the instructions cause the system to identify the sub-portion of the image based on identification of a region of the vehicle where a driver would likely be located.

12. The system of claim 8 , wherein the instructions cause the system to access a database or model that correlates the driver's body language detected by the body language component with one or more future vehicle movements.

13. The system of claim 12 , wherein the database or model correlates one or more of the following with the one or more future vehicle movements:
a waving motion of a hand;
a hand gesture comprising a palm facing toward a parent vehicle with fingers upward;
a gaze direction of the driver for a threshold period of time;
a series of head movements; and
a series of quick changes in gaze direction.

14. The system of claim 12 , wherein the database or model correlates a gaze direction with a future driving direction.

15. The system of claim 12 , wherein the database or model correlates a gesture with a current driving context, wherein the driving context comprises a stop at an intersection, an approach to an intersection, driving down a road with one or more nearby vehicles, merging onto a roadway, exiting a roadway, entering a parking lot or parking spot, or exiting a parking lot or parking spot.

16. The system of claim 8 , wherein the instructions cause the system to detect the driver's body language comprises using a neural network that determines the body language based on one the sub-region.

17. Non-transitory computer readable storage media storing instructions that, when executed by one or more processors, cause the processors to:
receive an image of a proximal vehicle;
identify a boundary around a region of the image where a driver of the proximal vehicle is located, wherein the instructions cause the one or more processors to identify the boundary based on identifying one or more windows of the proximal vehicle;
detect a body language of the driver by processing image data only in the boundary; and
predict future motion of the proximal vehicle based on the driver's body language.

18. The computer readable storage media of claim 17 , wherein the driver's body language comprises one or more of a driver's head orientation, a gaze direction, and a gesture, wherein the instructions further cause the processor to access a database or model that correlates one or more of the driver's head orientation, the gaze direction, and the gesture with one or more future vehicle movements.

19. The computer readable storage media of claim 17 , wherein the instructions further cause the processor to locate the proximal vehicle within the image.

20. The computer readable storage media of claim 17 , wherein the instructions further cause the one or more processor to determine a driving maneuver to be performed by a parent vehicle based on the predicted future motion.

Assignees

• Ford Global Tech Llc

Inventors

• Micks Ashley Elizabeth
• Banvait Harpreetsingh
• Jain Jinesh J
• Reiff Brielle

Classifications

• G06V20/597

  Recognising the driver's state or behaviour, e.g. attention or drowsiness · CPC title

• G06V40/20 Primary

  Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16) · CPC title

• G06V10/50

  by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis · CPC title

• B60W30/095

  Predicting travel path or likelihood of collision · CPC title

• B60W40/08 Primary

  related to drivers or passengers · CPC title