Incident site investigation and management support system based on unmanned aerial vehicles

Patent metadata

Field	Value
Publication number	US-11836858-B2
Application number	US-202117222927-A
Country	US
Kind code	B2
Filing date	Apr 5, 2021
Priority date	Aug 11, 2017
Publication date	Dec 5, 2023
Grant date	Dec 5, 2023

Abstract

Systems and methods allow for incident data collection and management system based on unmanned aerial vehicles (UAVs), that is, drones to help accelerate the data collection and analytics, information dissemination, and decision support at incident sites. The system architecture may include onsite, server, and offline components including flight planning subsystem, flight execution and mission control subsystem, information dissemination subsystem to travelers and traveler information services, the interface with traffic management center, and the data analytic, visualization, and training subsystems. Other embodiments include the video-based 3D incident site reconstruction methods, site positioning and scaling methods with pre-collected static background infrastructure data, data management and user charging methods, and training methods with the generated 3D model.

First claim

The invention claimed is:

1. A system for generating incident site data based on a reconstructed 3D model of the incident site and input data from one or more measurement tools, the system comprising one or more processors configured to perform the operations of:
a. causing generation of the reconstructed 3D model based on one of more object models of objects according to at the incident site and a current state of the objects;
b. causing generation of the incident site data based on:
(i) receipt of a high resolution (HR) line with point-to-point data, in 3D point cloud, selected with the one or more measurement tools being applied to an interface displaying a visualization of the reconstructed 3D model;
(ii) receipt of a high resolution (HR) area based on a drawn boundary and selected points, in 3D point cloud, selected with the one or more measurement tools being applied to the interface;
(iii) generation of a high resolution (HR) volume based on an undamaged object template in 3D point cloud;
(iv) generation of a high resolution (HR) object model based on a static 3D infrastructure model; and

c. identifying differences between volumes in the reconstructed 3D model and volumes in the high resolution (HR) object model to identify damage at the incident site.

2. The system of claim 1 , wherein the operation of generating the incident site data further comprises the operation of:
processing environmental data comprising one or more of: weather data, surrounding incident site environment data, collision roadside view data, collision driver view data, scenery data and illumination data.

3. The system of claim 1 , wherein the measurement input data comprises incident damage data from the one or more measurement tools being applied to at least one of: a tire mark, distance, vehicle geometry, impact area, volume, surface curvature in the interface displaying the visualization of the reconstructed 3D model.

4. The system of claim 1 , wherein generation of the high resolution (HR) object model based on a static 3-D infrastructure model comprises the operations of:
generating the static 3D infrastructure model;
generating a 3D model based on the high-resolution (HR) volume from one or more undamaged object templates, as 3D point cloud, representing an undamaged state of one or more objects at the incident site; and
generating the high resolution (HR) object model by merging the static infrastructure model with the 3D model.

5. The system of claim 4 , wherein generating the static 3D infrastructure model comprises the operations of:
accessing pre-collected infrastructure element data, the infrastructure element data comprising Mobile LiDAR-based geospatial indices for at least one of: a lane mark, a traffic sign, a light pole, a milepost, a traffic signal, an overhead gantry and a road object;
matching respective pre-collected infrastructure element data with infrastructure objects at the incident site; and
generating the static 3D infrastructure model based in part on the matched respective pre-collected infrastructure element data.

6. The system of claim 1 , wherein causing generation of the reconstructed 3D model based on one of more object models of objects according to at the incident site and a current state of the objects comprises the operation of:
creating one or more multi-resolution 3D models for different preview, site inspection and viewing, and site survey and measurement applications and different pricing levels by extracting and geotagging video frames at different rates.

7. The system of claim 6 , further comprising the operations of:
performing uplink to a traffic management center (TMC) by transmitting a video of the incident site collected by a drone to a video wall at the TMC, the uplink being through a video relaying service provided at a ground station with cellular communication or fiber communications;
performing uplink through edge devices (includes smartphone, tablets, laptops, or edge computers) to a video streaming service through cellular or fiber communications then transmitted to the traffic management center (TMC); and
performing downlink communications from the traffic management center (TMC) by establishing voice commands/communication through the drone or onsite emergency vehicles' microphone/speaker systems.

8. The system of claim 6 , further comprising the operations of:
flight planning for the drone to determine a flight plan, wherein the flight plan ensures a safety of onsite crew and traffic;
optimizing accuracy and efficiency of onsite data collection; and
reducing distraction of drivers in an environment external to the drone.

9. The system of claim 6 , further comprising the operations of:
performing Point-of-Interests (POI) orbits of multiple altitude levels centered around an accident site; and
allowing operators to draw the perimeters of accident sites, site crew activities, traffic control plan, and key obstacles on a satellite-map-based interface for designing flight path.

10. The system of claim 6 , further comprising the operations of:
detecting glare to mitigate the glare by optimizing a camera exposure setting to maximize a visibility of vehicle and lane marking edges; and
removing the glare by installing and adjusting an angle of polar lenses to the drone cameras based on a relationship between flight paths and a sun direction.

11. The system of claim 6 , further comprising the operations of:
crowdsourcing onsite and traffic conditions through feedbacks from operators and travelers around the incident site by allowing uploading of condition reports, pictures of the incident site and traffic, and other user-reported information.

12. The system of claim 6 , further comprising the operations of:
receiving uploaded information about an incident severity, traffic congestion, and traffic diversion to a cloud-based or TMC-server-based traveler information service through an event submission interface.

Assignees

• Univ Rutgers

Inventors

• Jin Jing
• Ouyang Bobby
• Ardestani Seyedamirali Mostafavizadeh
• Wang Yizhou
• Jiang Xiaowen
• Zhang Tianya

Classifications

• G08G5/80

  Anti-collision systems · CPC title

• G08G5/74

  for monitoring terrain · CPC title

• G08G5/57

  for unmanned aircraft · CPC title

• G08G5/56

  for two or more aircraft · CPC title

• G08G5/55

  for a single aircraft · CPC title