Detecting physical threats approaching a vehicle

Patent metadata

Field	Value
Publication number	US-10139827-B2
Application number	US-201615195827-A
Country	US
Kind code	B2
Filing date	Jun 28, 2016
Priority date	Jun 28, 2016
Publication date	Nov 27, 2018
Grant date	Nov 27, 2018

Abstract

The present invention extends to methods, systems, and computer program products for detecting physical threats approaching a vehicle. External sensors on a vehicle capture the environment around the vehicle. Approaching targets detected by the external sensors can be fed into a neural network to recognize and/or classify approaching targets as potential threats. Tracking mechanisms (e.g., Kalman filters, Particle filters, etc.) can leverage temporal information to determine if a threat is approaching a vehicle. When an approaching threat is detected, a vehicle can activate one or more counter measures to deter the threat. When a vehicle includes autonomous driving capabilities, counter measures can include automatically attempting to drive away from an approaching threat.

First claim

The invention claimed is:

1. A method for use at a vehicle, the method comprising:
determining that the path of a first object is likely to cause the first object and a corresponding second object to travel near the vehicle based on filtered sensor data from one or more sensors externally mounted on the vehicle;
providing the filtered sensor data as input to a neural network;
receiving a threat classification from the neural network classifying the first object and the corresponding second object collectively as a non-vehicular threat to an occupant of the vehicle;
indicating the threat classification in the vehicle cabin; and
automatically maneuvering the vehicle to get away from the first object and the second object.

2. The method of claim 1 , wherein receiving a threat classification comprises receiving an indication that the first object is a person and the corresponding second object is a weapon possessed by the person.

3. The method of claim 1 , wherein receiving a threat classification comprises receiving an indication that a weapon is approaching the vehicle.

4. The method of claim 1 , wherein receiving a threat classification comprises receiving an indication that the first object is a person and the corresponding second object is a mask worn by the person.

5. The method of claim 1 , further comprising activating counter measures to deter the non-vehicular threat.

6. The method of claim 1 , wherein receiving a threat classification comprises receiving an indication that the first object is a dog and the corresponding second object is the dog's teeth.

7. A method for use at a vehicle, the method comprising:
determining that the path of an object is likely to cause the object to travel near to the vehicle, including:
using one or more sensors mounted to the vehicle to monitor an area in proximity to the vehicle for approaching objects; and
filtering data from the one or more sensors on a heterogeneous computing platform at the vehicle to determine that the object has a speed and direction indicative of the object approaching space occupied by the vehicle;

providing the filtered data for the approaching object as input to a neural network;
receiving an indication from the neural network that the approaching object and a second object associated with the approaching object collectively represent a non-vehicular threat to a vehicle occupant based at least in part on identification of the second object; and
activating counter measures at the vehicle to address the non-vehicular threat, including automatically attempting to maneuver the vehicle to take the vehicle occupant away from both the approaching object and the second object.

8. The method of claim 7 , wherein activating counter measures at the vehicle to address the non-vehicular threat comprises automatically making a phone call to emergency services.

9. The method of claim 7 , wherein receiving an indication from the neural network that the approaching object and a second object associated with the approaching object collectively represents a non-vehicular threat comprises receiving a threat probability from the neural network.

10. The method of claim 7 , wherein using one or more sensors mounted to the vehicle to monitor an area in proximity to the vehicle for approaching objects comprises using at least two of: a camera, a LIDAR sensor, and a sound based object detector to monitor an area in proximity to the vehicle for approaching objects.

11. The method of claim 10 , wherein filtering data from the one or more sensors on a heterogeneous computing platform comprises filtering data from the plurality of sensors on at least two of: a Central Processing Unit (CPU), a Graphical Processing Unit (GPU), and a Field Programmable Gate Array (FPGA) in the vehicle.

12. The method of claim 7 , wherein filtering data from the one or more sensors comprises filtering the data using one or more of: a Kalman filter and a Particle Filter.

13. The method of claim 7 , wherein filtering data from the one or more sensors comprises determining an optical flow of the non-vehicular threat.

14. The method of claim 7 , wherein receiving an indication from the neural network that the approaching object and a second object associated with the approaching object collectively represent a non-vehicular threat to the vehicle occupant comprises receiving an indication that the approaching object is a person and the associated second object is one of: a weapon in possession of the person or a mask in possession of the person.

15. The method of claim 7 , wherein activating counter measures at the vehicle comprises activating counter measures including one or more of: recording the approaching object, locking the doors of the vehicle, issuing an audible warning in the vehicle cabin, honking a horn of the vehicle, or flashing lights of the vehicle.

16. The method of claim 1 , further comprising automatically activating counter measures at the vehicle including one or more of: recording the first object, locking the doors of the vehicle, calling emergency services, issuing an audible warning in the vehicle cabin, honking a horn of the vehicle, or flashing lights of the vehicle.

17. A vehicle, the vehicle comprising:
one or more externally mounted sensors for monitoring an area in proximity to the vehicle;
one or more processors;
system memory coupled to one or more processors, the system memory storing instructions that are executable by the one or more processors;
the one or more processors configured to execute the instructions stored in the system memory to:
determine that the path of an object is likely to cause the object and a second object to travel near to the vehicle, including:
use the one or more externally mounted sensors to monitor the area in proximity to the vehicle for approaching objects; and
filter data from the one or more sensors to determine that the object has a speed and direction indicative of the object approaching space occupied by the vehicle;

provide the filtered data as input to a neural network;
receive a threat probability from the neural network indicating a probability that the object and the second object collectively represent a non-vehicular threat to a vehicle occupant; and
control one or more vehicle components, wherein the one or more vehicle components maneuver the vehicle to get away from both the object and the second object.

18. The vehicle of claim 17 , wherein the one or more externally mounted sensors include one or more of: a camera, a LIDAR sensor, a RADAR sensor, and an ultrasonic sensor.

19. The vehicle of claim 17 , wherein the one or more processors configured to execute the instructions to receive an indication from the neural network that the approaching object represents a non-vehicular threat to the vehicle occupants comprises the one or more processors configured to execute the instructions to receive an indication that the approaching object is a person and the associated second object is one of: a weapon in possession of the person or a mask in possession of the person.

20. The vehicle of claim 17 , wherein the one or more processors configured to execute the instructions to activate counter measures at the vehicle comprises the one or more processors configured to execute the instructions to activate counter measures including one or more of: recording the object, locking the doors of the vehicle, issuing an audible warning in the vehicle cabin, honking a horn of the vehicle, or flashing lights of the vehicle.

Assignees

• Ford Global Tech Llc

Inventors

• Charette Francois
• Myers Scott Vincent
• Scaria Lisa
• Gurghian Alexandru Mihai

Classifications

• G05D2201/0213

  Physics · mapped topic

• B60W50/14

  Means for informing the driver, warning the driver or prompting a driver intervention · CPC title

• B60W2050/143

  Alarm means · CPC title

• G05D1/0088

  characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00) · CPC title

• B60W10/30

  including control of auxiliary equipment, e.g. air-conditioning compressors or oil pumps · CPC title