Sensor fusion by operations-control vehicle for commanding and controlling autonomous vehicles

Patent metadata

Field	Value
Publication number	US-10928826-B2
Application number	US-201816172648-A
Country	US
Kind code	B2
Filing date	Oct 26, 2018
Priority date	Oct 26, 2018
Publication date	Feb 23, 2021
Grant date	Feb 23, 2021

Abstract

In one embodiment, a method includes receiving sensor data from one or more sensors of each of one or more vehicles, processing a combination of the sensor data to generate an assessment of an area surrounding the one or more vehicles based on one or more points-of-view of the area, the one or more points-of-view of the area generated based on synchronizing the combination of the sensor data, and detecting an occurrence of an event based on the assessment of the area. The method further includes identifying one or more instructions corresponding to the event, the instructions associated with a particular vehicle and sending one or more executable instructions based on the instructions to the particular vehicle.

First claim

What is claimed is:

1. A method comprising:
receiving, by a computing device, a plurality of sensor data from a plurality of vehicles within a geographic area that are associated with each other, wherein the plurality of sensor data are associated with different points-of-view of the geographic area;
processing, by the computing device, a combination of the plurality of sensor data from the plurality of vehicles to generate an assessment of the geographic area surrounding the plurality of vehicles based on the different points-of-view of the geographic area;
detecting, by the computing device, an occurrence of an event based on the assessment of the geographic area;
identifying, by the computing device, a particular vehicle in the plurality of vehicles that is predicted to be affected by the occurrence of the event; and
sending, by the computing device, one or more instructions associated with the occurrence of the event to the particular vehicle.

2. The method of claim 1 , further comprising, prior to receiving the plurality of sensor data:
requesting, by the computing device, the plurality of sensor data from the plurality of vehicles.

3. The method of claim 1 , wherein the plurality of sensor data are automatically sent from each of the plurality of vehicles (1) when the vehicle determines that an event has occurred or (2) periodically based on a predetermined time period.

4. The method of claim 1 , wherein processing the combination of the plurality of sensor data from the plurality of vehicles comprises:
associating each of the plurality of sensor data with one or more time stamps;
synchronizing the plurality of sensor data based on the time stamps;
generating the different points-of-view of the geographic area based on the synchronized plurality of sensor data; and
assessing the different points-of-view of the geographic area to determine the occurrence of the event.

5. The method of claim 1 , wherein identifying the particular vehicle in the plurality of vehicles that is predicted to be affected by the occurrence of the event comprises:
determining one or more of the plurality of vehicles that are affected by, or predicted to be affected by, the occurrence of the event;
prioritizing the determined one or more vehicles based on the occurrence of the event; and
selecting a vehicle of the determined one or more vehicles with a highest priority as the particular vehicle.

6. The method of claim 5 , wherein prioritizing the determined one or more vehicles is further based on at least one of:
a degree of route alteration associated with a vehicle of the determined one or more vehicles due to the occurrence of the event,
a distance from a vehicle of the determined one or more vehicles to a location of the event,
a current route of a vehicle of the determined one or more vehicles, or
a distance from a vehicle of the determined one or more vehicles to a location associated with the computing device.

7. The method of claim 1 , wherein detecting the occurrence of the event based on the assessment of the geographic area comprises:
determining one or more relevant routes associated with the plurality of vehicles; and
analyzing the assessment of the geographic area based on the one or more relevant routes associated with the plurality of vehicles.

8. The method of claim 1 , wherein the one or more instructions sent to the particular vehicle instruct the particular vehicle to make one or more route alterations.

9. The method of claim 1 , wherein the one or more instructions sent to the particular vehicle instruct the particular vehicle to process and assess additional sensor data from one or more other vehicles.

10. The method of claim 1 , further comprising:
assessing, by the computing device, a state of one or more sensors associated with each of the plurality of vehicles based on a comparison of the plurality of sensor data from the plurality of vehicles; and
determining, by the computing device, that one or more of the sensors associated with the plurality of vehicles require calibration based on the comparison.

11. The method of claim 1 , wherein the computing device resides in, is coupled to, or is located within a vicinity of one of the plurality of vehicles, an aerial vehicle, or a base station.

12. The method of claim 1 , wherein the event is a predetermined event, and wherein the one or more instructions associated with the event are predetermined instructions.

13. A system comprising: one or more processors and one or more computer-readable non-transitory storage media coupled to one or more of the processors, the one or more computer-readable non-transitory storage media comprising instructions operable when executed by one or more of the processors to cause the system to perform operations comprising:
receiving a plurality of sensor data from a plurality of vehicles within a geographic area that are associated with each other, wherein the plurality of sensor data are associated with different points-of-view of the geographic area;
processing a combination of the plurality of sensor data from the plurality of vehicles to generate an assessment of the geographic area surrounding the plurality of vehicles based on the different points-of-view of the geographic area;
detecting an occurrence of an event based on the assessment of the geographic area;
identifying a particular vehicle in the plurality of vehicles that is predicted to be affected by the occurrence of the event; and
sending one or more instructions associated with the occurrence of the event to the particular vehicle.

14. The system of claim 13 , wherein processing the combination of the plurality of sensor data from the plurality of vehicles comprises:
associating each of the plurality of sensor data with one or more time stamps;
synchronizing the plurality of sensor data based on the time stamps;
generating the different points-of-view of the geographic area based on the synchronized plurality of sensor data; and
assessing the different points-of-view of the geographic area to determine the occurrence of the event.

15. The system of claim 13 , wherein identifying the particular vehicle in the plurality of vehicles that is predicted to be affected by the occurrence of the event comprises:
determining one or more of the plurality of vehicles that are affected by, or predicted to be affected by, the occurrence of the event;
prioritizing the determined one or more vehicles based on at least one of:
a degree of route alteration associated with a vehicle of the determined one or more vehicles due to the occurrence of the event,
a distance from a vehicle of the determined one or more vehicles to a location of the event,
a current route of a vehicle of the determined one or more vehicles, or
a distance from a vehicle of the determined one or more vehicles to a location associated with the computing device; and

selecting a vehicle of the determined one or more vehicles with a highest priority as the particular vehicle.

16. The system of claim 13 , wherein detecting the occurrence of the event based on the assessment of the geographic area comprises:
determining one or more relevant routes associated with the plurality of vehicles; and
analyzing the assessment of the geographic area based on the one or more relevant routes associated with the plurality of vehicles.

17. One or more computer-readable non-transitory storage media including instructions that, when executed by one or more processors, are configured to cause the one or more processors to perform operatio

Assignees

• Lyft Inc

Inventors

• Abari Farzad Cyrus Foroughi
• Clément Romain
• Paine Roxie Lynn

Classifications

• G01S13/931 Primary

  of land vehicles · CPC title

• G05D1/0231 Primary

  using optical position detecting means (position-fixing by using electromagnetic waves other than radio waves, e.g. optical position detecting means G01S5/16) · CPC title

• G01S13/867

  Combination of radar systems with cameras · CPC title

• G01S13/865

  Combination of radar systems with lidar systems · CPC title

• G05D1/0088

  characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00) · CPC title