Detecting objects obstructing a driver's view of a road

Patent metadata

Field	Value
Publication number	US-9568611-B2
Application number	US-201514830873-A
Country	US
Kind code	B2
Filing date	Aug 20, 2015
Priority date	Aug 20, 2014
Publication date	Feb 14, 2017
Grant date	Feb 14, 2017

Abstract

A system and method for a motorized land vehicle that detects objects obstructing a driver's view of an active road, includes an inertial measurement unit-enabled global position system (GPS/IMU) subsystem for obtaining global position system (GPS) position and heading data of a land vehicle operated by the driver as the vehicle travels along a road, a street map subsystem for obtaining street map data of the GPS position of the vehicle using the GPS position and heading data as the vehicle travels along the road, and a three-dimensional (3D) object detector subsystem for detecting objects ahead of the vehicle and determining a 3D position and 3D size data of each of the detected objects ahead of the vehicle. The street map subsystem merges the street map data, the GPS position and heading data of the vehicle and the 3D position data and 3D size data of the detected objects, to create real-time two-dimensional (2D) top-view map representation of a traffic scene ahead of the vehicle. The street map subsystems finds active roads ahead of the vehicle in the traffic scene, and finds each active road segment of the active roads ahead of the vehicle that is obstructed by one of the detected objects. A driver alert subsystem notifies a driver of the vehicle of each of the active road segments that is obstructed by one of the detected objects.

First claim

What is claimed is:

1. A system for a motorized land vehicle that detects objects obstructing a driver's view of an active road, the system comprising:
an inertial measurement unit-enabled global position system (GPS/IMU) subsystem for obtaining global position system (GPS) position and heading data of a land vehicle operated by the driver as the vehicle travels along a road;
a street map subsystem for obtaining street map data of the GPS position of the vehicle using the GPS position and heading data as the vehicle travels along the road;

a three-dimensional (3D) object detector subsystem for detecting objects ahead of the vehicle and determining a 3D position data of each of the detected objects ahead of the vehicle; and
a driver alert subsystem;
wherein the street map subsystem merges the street map data, the GPS position and heading data of the vehicle and the 3D position data of the detected objects, to create real-time two-dimensional (2D) top-view representation of a traffic scene ahead of the vehicle, finds active roads ahead of the vehicle in the traffic scene, and finds each active road segment of the active roads ahead of the vehicle that is obstructed by one of the detected objects;

wherein the driver alert subsystem notifies the driver of the vehicle of each of the active road segments that is obstructed by one of the detected objects;
wherein the 3D object detector subsystem further determines a 3D size of each of the detected objects ahead of the vehicle; and
wherein the 3D object detector subsystem includes at least one video camera and a neural network, the at least one video camera providing the 3D position data of each of the detected objects from motion data in a video stream of the traffic scene obtained therewith and the neural network providing the type and scale of the object from the video stream, wherein the 3D size of each of the objects is determined from the type and scale of the object combined with approximate standard sizes appropriate of the recognized type of object.

2. A system for a motorized land vehicle that detects objects obstructing a driver's view of an active road, the system comprising:
an inertial measurement unit-enabled global position system (GPS/IMU) subsystem for obtaining global position system (GPS) position and heading data of a land vehicle operated by the driver as the vehicle travels along a road;
a street map subsystem for obtaining street map data of the GPS position of the vehicle using the GPS position and heading data as the vehicle travels along the road;
a three-dimensional (3D) object detector subsystem for detecting objects ahead of the vehicle and determining a 3D position data of each of the detected objects ahead of the vehicle; and
a driver alert subsystem;
wherein the street map subsystem merges the street map data, the GPS position and heading data of the vehicle and the 3D position data of the detected objects, to create real-time two-dimensional (2D) top-view representation of a traffic scene ahead of the vehicle, finds active roads ahead of the vehicle in the traffic scene, and finds each active road segment of the active roads ahead of the vehicle that is obstructed by one of the detected objects;
wherein the driver alert subsystem notifies the driver of the vehicle of each of the active road segments that is obstructed by one of the detected objects;
wherein the 3D object detector subsystem further determines a 3D size of each of the detected objects ahead of the vehicle; and

wherein the 3D object detector subsystem includes radar, at least one video camera, and a neural network, the radar providing two-dimensional (2D) position and speed data of each of the detected objects, the at least one video camera providing a video stream of the traffic scene obtained therewith and the neural network providing the type and scale of the object from the video stream, wherein the 3D size of each of the objects is determined from the type and scale of the object combined with approximate standard sizes appropriate of the recognized type of object.

3. The system of claim 1 , wherein the 3D object detector subsystem includes lidar providing 3D position and size of each of the detected objects.

4. The system of claim 3 , wherein the 3D object detector subsystem further includes at least one video camera, and a neural network, providing a video stream of the traffic scene obtained therewith and the neural network providing the type of the object from the video stream, wherein the type of the object is used to verify and validate lidar information.

5. The system of claim 1 , wherein the GPS/IMU subsystem and the street map subsystem comprise a GPS map unit.

6. The system of claim 1 , wherein the GPS/IMU comprises a GPS chip and the street map data is provided by an Internet provider.

7. The system of claim 1 , wherein the merging of the street map data, the GPS position and heading data of the vehicle and the 3D position data of the detected objects includes converting street map coordinates into relative distance from the vehicle's center of mass, making the 3D positions of the objects relative to the GPS/IMU subsystem, and projecting along a z axis a bounding box of the 3D object onto the street map.

8. The system of claim 1 , wherein the finding of the active roads ahead of the vehicle in the traffic scene is determined with the 3D position, speed and the heading of the vehicle, wherein based on the vehicle's position, the closest road on the street map is selected, wherein based on the vehicle's speed, a proportional active distance ahead of the selected road is calculated and a corresponding segment of the road is deemed active and further searched for intersecting roads, wherein for each intersection found, the roads leading to the intersection become active.

9. The system of claim 1 , wherein the finding of each active road segment of the active roads ahead of the vehicle that is obstructed by one of the detected objects comprises tracing rays from the drivers' eyes that pass by comers of each of the detected objects, and wherein segments formed by intersecting the rays with the active roads are hidden road segments obstructed by one of the detected objects.

10. The system of claim 1 , wherein the driver alert subsystem generates an audio-locating sound at a virtual position of the obstructing vehicle thereby alerting the driver.

11. The system of claim 1 , wherein the driver alert subsystem overlays a 3D bounding box of each of the obstructing objects onto a live video feed of the road ahead.

12. A system for a motorized land vehicle that detects objects obstructing a driver's view of an active road, the system comprising:
an inertial measurement unit-enabled global position system (GPS/IMU) subsystem for obtaining global position system (GPS) position and heading data of a land vehicle operated by the driver as the vehicle travels along a road;
a street map subsystem for obtaining street map data of the GPS position of the vehicle using the GPS position and heading data as the vehicle travels along the road;
a three-dimensional (3D) object detector subsystem for detecting objects ahead of the vehicle and determining a 3D position data of each of the detected objects ahead of the vehicle; and
a driver alert subsystem;
wherein the street map subsystem merges the street map data, the GPS position and heading data of the vehicle and the 3D position data of the detected objects, to create real-time two-dimensional (2D) top-view representation of a traffic scene ahead of the vehicle, finds active roads ahead of the vehicle in the traffic scene, and finds each active road segment of the ac

Assignees

• Nec Lab America Inc
• Nec Corp

Inventors

• Cosatto Eric
• Graf Hans Peter

Classifications

• G01C21/3629

  Guidance using speech or audio output, e.g. text-to-speech (text to speech systems per se G10L13/00) · CPC title

• G01S13/931

  of land vehicles · CPC title

• G01S17/89

  for mapping or imaging · CPC title

• G01S19/51 Primary

  Relative positioning · CPC title

• G01S19/14

  specially adapted for specific applications · CPC title