Autonomous driving at intersections based on perception data

Patent metadata

Field	Value
Publication number	US-9983591-B2
Application number	US-201514933693-A
Country	US
Kind code	B2
Filing date	Nov 5, 2015
Priority date	Nov 5, 2015
Publication date	May 29, 2018
Grant date	May 29, 2018

Abstract

Systems, methods, and devices for predicting a driver's intention and future movements of a proximal vehicle, whether an automated vehicle or a human driven vehicle, are disclosed herein. A system for predicting future movements of a vehicle includes an intersection component, a camera system, a boundary component, and a prediction component. The intersection component is configured to determine that a parent vehicle is near an intersection. The camera system is configured to capture an image of the proximal vehicle. The boundary component is configured to identify a sub-portion of the image containing a turn signal indicator on the proximal vehicle. The prediction component is configured to predict future movement of the proximal vehicle through the intersection based on a state of the turn signal indicator.

First claim

What is claimed is:

1. A system comprising:
an intersection component configured to determine that a parent vehicle is near an intersection;
a camera system configured to capture an image of a proximal vehicle;
a boundary component configured to identify a sub-portion of the image containing a turn signal indicator on the proximal vehicle and a driver of the proximal vehicle;
a turn signal component configured to process image data in the sub-portion of the image to determine the state of the turn signal indicator;
a prediction component configured to predict future movement of the proximal vehicle through the intersection based on a state of the turn signal indicator; and
at least one or more processors configured to:
process image data in the sub-portion of the image containing the driver of the proximal vehicle to determine a body language of the driver;
predict future movement of the proximal vehicle through the intersection based on the state of the turn signal indicator and the body language of the driver;
determine a time for the parent vehicle to proceed through the intersection based on a predicted future movement of the proximal vehicle; and
wherein the at least one or more processors or an actuator causes the parent vehicle to perform a driving maneuver based on a determined time to proceed through the intersection.

2. The system of claim 1 , further comprising a previous state component configured to determine one or more previous states of the proximal vehicle based on wireless communications indicating the one or more previous states of the proximal vehicle, wherein the prediction component is configured to predict future movements of the proximal vehicle based on the one or more previous states of the proximal vehicle.

3. The system of claim 2 , wherein the wireless communication comprises one or more of a vehicle-to-vehicle (V2V) communication and a vehicle-to-infrastructure (V2X) communication.

4. The system of claim 2 , wherein the one or more previous states indicate a duration of time during which the proximal vehicle has been located near the intersection.

5. The system of claim 1 , further comprising a vehicle movement component configured to determine one or more vehicle movements of the proximal vehicle, wherein the prediction component is configured to predict future movements of the proximal vehicle based on the one or more vehicle movements of the proximal vehicle.

6. The system of claim 1 , wherein the boundary component is further configured to identify a sub-portion of the image corresponding to a location of a driver, the system further comprising a body language component configured to detect a driver's body language by identifying one or more of a driver's head orientation, and a gaze direction, wherein the prediction component is configured to predict future movements of the proximal vehicle based on the driver's body language.

7. A computer implemented method comprising:
receiving an image of a proximal vehicle near an intersection and storing the image in computer memory;
identifying, using one or more processors, a sub-portion of the image containing a turn signal indicator on the proximal vehicle;
identifying, using the one or more processors, a sub-portion of the image containing a driver of the proximal vehicle;
processing, using the one or more processors, image data in the sub-portion of the image containing a turn signal indicator to determine the state of the turn signal indicator;
processing, using the one or more processors, image data in the sub-portion of the image containing a driver of the proximal vehicle to determine a body language of the driver;
predicting, using the one or more processors, future movement of the proximal vehicle through the intersection based on the state of the turn signal indicator and the body language of the driver;
determining, using the one or more processors, a time for a parent vehicle to proceed through the intersection based on a predicted future movement of the proximal vehicle; and
causing, using one or more processors or actuators, the parent vehicle to perform a driving maneuver based on a determined time to proceed through the intersection.

8. The method of claim 7 , further comprising determining that the parent vehicle is near an intersection.

9. The method of claim 7 , wherein determining the body language of the driver comprises identifying one or more of a driver's head orientation, a gaze direction, and a gesture.

10. The method of claim 7 , further comprising determining one or more previous states of the proximal vehicle based on a wireless communication, wherein predicting future movements of the proximal vehicle comprises predicting based on the one or more previous states of the proximal vehicle.

11. The method of claim 10 , wherein the wireless communication comprises one or more of a vehicle-to-vehicle (V2V) communication and a vehicle-to-infrastructure (V2X) communication.

12. The method of claim 7 , further comprising detecting one or more vehicle movements of the proximal vehicle, wherein predicting future movements of the proximal vehicle comprises predicting based on the one or more vehicle movements of the proximal vehicle.

13. The method of claim 7 , further comprising accessing or processing a model or database correlating the state of the turn signal indicator and the body language of the driver with the predicted future movement.

14. The method of claim 13 , wherein the model or database comprises machine learning values or correlations based on motion of one or more vehicles, driver body language, and turn signal information during previous intersection driving scenarios.

15. Computer readable non-transitory storage media storing instructions that, when executed by one or more processors, cause the processors to:
determine that a parent vehicle is near an intersection;
capture a plurality of images of a proximal vehicle;
identify a sub-portion in each of the plurality of the images containing a turn signal indicator on the proximal vehicle;
process image data in the sub-portion of the plurality of images to determine the state of the turn signal indicator;
predict future movement of the proximal vehicle through the intersection based on a state of the turn signal indicator;
process, using the one or more processors, image data in the sub-portion of the image containing a driver of the proximal vehicle to determine a body language of the driver;
predict, using the one or more processors, future movement of the proximal vehicle through the intersection based on the state of the turn signal indicator and the body language of the driver;
determine, using the one or more processors, a time for a parent vehicle to proceed through the intersection based on a predicted future movement of the proximal vehicle; and
cause, using one or more processors or actuators, the parent vehicle to perform a driving maneuver based on a determined time to proceed through the intersection.

16. The computer readable storage media of claim 15 , wherein the instructions further cause the processor to:
determine one or more previous states of the proximal vehicle based on wireless communications indicating the one or more previous states of the proximal vehicle; and
predict future movements of the proximal vehicle based on the one or more previous states of the proximal vehicle.

17. The computer readable storage media of claim 15 , wherein the instructions further cause the processor to:
identify a sub-portion of the image corresponding to a location of a driver;

Assignees

• Ford Global Tech Llc

Inventors

• Micks Ashley Elizabeth
• Banvait Harpreetsingh
• Jain Jinesh J
• Reiff Brielle

Classifications

• G06V20/58

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title

• G06V20/584

  of vehicle lights or traffic lights · CPC title

• G06V40/161

  Detection; Localisation; Normalisation · CPC title

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• G06V10/25

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title