Emergency vehicle detection and response

Patent metadata

Field	Value
Publication number	US-12505739-B2
Application number	US-202217947561-A
Country	US
Kind code	B2
Filing date	Sep 19, 2022
Priority date	Dec 31, 2019
Publication date	Dec 23, 2025
Grant date	Dec 23, 2025

Abstract

Techniques for detecting and responding to an emergency vehicle are discussed. A vehicle computing system may determine that an emergency vehicle based on sensor data, such as audio and visual data. In some examples, the vehicle computing system may determine aggregate actions of objects (e.g., other vehicles yielding) proximate the vehicle based on the sensor data. In such examples, a determination that the emergency vehicle is operating may be based on the actions of the objects. The vehicle computing system may, in turn, identify a location to move out of a path of the emergency vehicle (e.g., yield) and may control the vehicle to the location. The vehicle computing system may determine that the emergency vehicle is no longer relevant to the vehicle and may control the vehicle along a route to a destination. Determining to yield and/or returning to a mission may be confirmed by a remote operator.

First claim

What is claimed is:

1 . A system comprising:
one or more processors; and one or more computer-readable media storing instructions executable by the one or more processors, wherein the instructions, when executed, cause the system to perform operations comprising:
receiving sensor data from a sensor associated with an autonomous vehicle traversing an environment;
determining, based at least in part on the sensor data, a traffic flow associated with objects operating in the environment;
determining a difference between the traffic flow and an expected traffic flow;
based at least in part on determining that the difference meets or exceeds a threshold, determining that an emergency vehicle is operating in the environment;
determining, based at least in part on the sensor data, a predicted trajectory associated with the emergency vehicle;
determining, based at least in part on the predicted trajectory, an action of the autonomous vehicle;
determining, based on the action, a trajectory for performing the action;
determining, based on a current trajectory of the autonomous vehicle and the trajectory, an acceleration needed to switch to the trajectory from the current trajectory of the autonomous vehicle;
determining that the acceleration is equal to or less than a threshold acceleration; and
based on determining that the acceleration is equal to or less than the threshold acceleration, controlling the autonomous vehicle based on the trajectory.

2 . The system of claim 1 , the operations further comprising:
determining, based at least in part on the acceleration, a cost associated with the trajectory, the cost being indicative of at least comfort of a passenger of the autonomous vehicle, wherein the controlling the autonomous vehicle is determined based at least in part on the cost.

3 . The system of claim 2 , wherein the cost is further based at least in part on one or more of: safety of the autonomous vehicle, progress toward a destination, or traffic rules.

4 . The system of claim 1 , the operations further comprising:
determining, based at least in part on the sensor data, that an additional vehicle is operating in the environment; determining a predicted trajectory of the additional vehicle; and determining a deviation of the additional vehicle from the predicted trajectory of the additional vehicle, wherein the determining the action of the autonomous vehicle is based at least in part on the deviation of the additional vehicle.

5 . The system of claim 1 , the operations further comprising:
determining a likelihood that the emergency vehicle will impact motion of the autonomous vehicle; sending, based at least on the likelihood, at least one of the sensor data or data based on the sensor data to a remote computing device; and receiving, from the remote computing device, an indication that the emergency vehicle is operating in the environment, wherein determining that the emergency vehicle is operating in the environment is based at least in part on the indication.

6 . The system of claim 1 , wherein the action comprises at least one of: a pull-over action, a lane-change action, a yielding action, a slowing action, or a turning action.

7 . The system of claim 1 , the operations further comprising:
receiving additional sensor data; determining, based at least in part on the additional sensor data, that other vehicles operating in the environment have started movement; and based on determining that the other vehicles have started movement, controlling the autonomous vehicle on a continuing trajectory toward a destination.

8 . The system of claim 1 , the operations further comprising: determining, based at least in part on the sensor data, a motion trend of objects operating in the environment; and determining that a difference between the motion trend and an expected motion trend meets or exceeds a threshold, wherein determining that the emergency vehicle is operating in the environment is further based at least in part on the difference meeting or exceeding the threshold.

9 . The system of claim 1 , the operations further comprising:
determining a yielding location associated with a drivable surface in the environment, wherein the yielding location is outside the predicted trajectory associated with the emergency vehicle; and determining, based at least in part on the current trajectory and the predicted trajectory, a yielding trajectory to the yielding location, wherein controlling the autonomous vehicle comprises yielding to the emergency vehicle following the yielding trajectory.

10 . A method comprising:
receiving a vehicle trajectory associated with a vehicle traversing an environment; receiving sensor data associated with the environment; determining, based at least in part on the sensor data, a motion trend of objects operating in the environment; determining that a difference between the motion trend and an expected motion trend meets or exceeds a threshold; determining, based at least in part on determining that the difference meets or exceeds the threshold, that an emergency vehicle is operating in the environment; determining, based at least in part on the vehicle trajectory and a trajectory associated with the emergency vehicle, a yielding trajectory to yield to the emergency vehicle; determining a first cost associated with the vehicle continuing on the vehicle trajectory and a second cost associated with the vehicle following the yielding trajectory; and controlling, based on comparing the first cost and the second cost, the vehicle on one of the vehicle trajectory or the yielding trajectory.

11 . The method of claim 10 , wherein controlling the vehicle on the yielding trajectory comprises:
determining a yielding location associated with a drivable surface in the environment; determining that the yielding location is outside a predicted trajectory of the emergency vehicle; and determining that an object is not located at the yielding location.

12 . The method of claim 10 , wherein determining that the emergency vehicle is operating in the environment further comprises:
determining that an audio portion of the sensor data is associated with an approaching emergency vehicle; determining that a visual portion of the sensor data is associated with at least one of a blue or a red light; and determining, based at least in part on the audio portion and the visual portion of the sensor data, that a classification of a first detected object comprises the emergency vehicle.

13 . The method of claim 10 , wherein the first cost or the second cost is based at least in part on one or more of: a likelihood that the emergency vehicle will impact motion of the vehicle, comfort of a passenger of the vehicle, or vehicle operation rules.

14 . The method of claim 13 , wherein the likelihood is based on at least one of:
receiving, from a remote computing device associated with a remote operator, an indication that the emergency vehicle is irrelevant to the vehicle; determining that the trajectory associated with the emergency vehicle is indicative of irrelevance to the vehicle; determining that a direction of arrival of an audio signal associated with the emergency vehicle is indicative of irrelevance to the vehicle; or determining that an average direction of movement associated with one or more objects in the environment is indicative of the emergency vehicle being irrelevant to the vehicle.

15 . The method of claim 10 , further comprising:
receiving additional sensor data; determining, based at least in part on the additional

Assignees

• Zoox Inc

Inventors

• Tariq Sarah
• Gogna Ravi
• Wimmershoff Marc
• Subasingha Subasingha Shaminda

Classifications

• B60W2420/408

  Radar; Laser, e.g. lidar · CPC title

• B60W2420/403

  Image sensing, e.g. optical camera · CPC title

• G08G1/017

  identifying vehicles (G08G1/015, G08G1/054 take precedence) · CPC title

• B60W2754/20

  Lateral distance · CPC title

• B60W2754/70

  Relative heading · CPC title