Vehicular control system with handover procedure for driver of controlled vehicle

Patent metadata

Field	Value
Publication number	US-12321173-B2
Application number	US-202418600958-A
Country	US
Kind code	B2
Filing date	Mar 11, 2024
Priority date	Sep 29, 2016
Publication date	Jun 3, 2025
Grant date	Jun 3, 2025

Abstract

A vehicular control system includes a forward-viewing camera, a forward-sensing sensor and a cabin-sensing imaging sensor. With the system controlling driving of the vehicle, the system determines a triggering event that triggers handing over driving of the vehicle to a driver of the vehicle before the vehicle encounters an event point associated with the triggering event. The vehicular control system (i) determines a total action time available before the vehicle encounters the event point, (ii) estimates a driver takeover time for the driver to take over control of the vehicle and (iii) estimates a handling time for the driver to control the vehicle to avoid encountering the event point. The vehicular control system estimates the driver takeover time based at least in part on processing by the vehicular control system of the in-cabin image data captured by the cabin-sensing imaging sensor.

First claim

The invention claimed is:

1. A vehicular control system, the vehicular control system comprising:
a forward-viewing camera disposed at a windshield of a vehicle equipped with the vehicular control system, the forward-viewing camera viewing through the windshield forward of the vehicle, the forward-viewing camera capturing image data;
at least one forward-sensing sensor disposed at the vehicle and sensing forward of the vehicle, the at least one forward-sensing sensor capturing sensor data;
a cabin-sensing imaging sensor disposed in a cabin of the vehicle and sensing interior of the cabin of the vehicle, the cabin-sensing imaging sensor capturing in-cabin image data;
an electronic control unit (ECU) comprising electronic circuitry, the electronic circuitry of the ECU processing (i) the image data captured by the forward-viewing camera and (ii) the sensor data captured by the at least one forward-sensing sensor;
wherein the vehicular control system is operable to control driving of the vehicle at least in part responsive to processing by the vehicular control system of (i) the image data captured by the forward-viewing camera, (ii) the sensor data captured by the at least one forward-sensing sensor and (iii) the in-cabin image data captured by the cabin-sensing imaging sensor;
wherein, with the vehicular control system controlling driving of the vehicle in, and responsive at least in part to processing at the ECU of the image data captured by the forward-viewing camera and the sensor data captured by the at least one forward-sensing sensor, the vehicular control system determines a triggering event that triggers handing over driving of the vehicle to a driver of the vehicle before the vehicle encounters an event point associated with the triggering event;
wherein the triggering event comprises a hazardous condition ahead of the vehicle that is hazardous to driving of the vehicle, and wherein the event point is the location of the hazardous condition;
wherein, responsive at least in part to determination of the triggering event, the vehicular control system (i) determines a total action time available before the vehicle encounters the event point, (ii) estimates a driver takeover time for the driver to take over control of the vehicle and (iii) estimates a handling time for the driver to control the vehicle to avoid encountering the event point; and
wherein the vehicular control system estimates the driver takeover time based at least in part on processing by the vehicular control system of the in-cabin image data captured by the cabin-sensing imaging sensor.

2. The vehicular control system of claim 1 , wherein, responsive to the vehicular control system requesting the driver take over control of the vehicle, and responsive to the driver not taking over control of the vehicle within a time period, the vehicular control system maneuvers the vehicle to a stopping location.

3. The vehicular control system of claim 2 , wherein the time period constitutes the difference between the determined total action time and the estimated handling time.

4. The vehicular control system of claim 2 , wherein the stopping location comprises an emergency stopping lane of a road that the vehicle is travelling along.

5. The vehicular control system of claim 2 , wherein the stopping location comprises a shoulder of a road that the vehicle is travelling along.

6. The vehicular control system of claim 2 , wherein the stopping location comprises a parking bay.

7. The vehicular control system of claim 2 , wherein the stopping location comprises a parking lot.

8. The vehicular control system of claim 1 , wherein, responsive to the vehicular control system determining that the estimated driver takeover time is less than the difference between the determined total action time and the estimated handling time, control of the vehicle is handed over to the driver of the vehicle.

9. The vehicular control system of claim 1 , wherein, responsive to the vehicular control system determining that the estimated driver takeover time is greater than the difference between the determined total action time and the estimated handling time, the vehicular control system maneuvers the vehicle to a stopping location.

10. The vehicular control system of claim 1 , wherein the at least one forward-sensing sensor comprises at least one selected from the group consisting of (i) a forward-sensing radar sensor and (ii) a forward-sensing lidar sensor.

11. The vehicular control system of claim 1 , wherein the vehicular control system estimates the driver takeover time for the driver to take over control of the vehicle based at least in part on monitoring of driver attentiveness via processing by the vehicular control system of the in-cabin image data captured by the cabin-sensing imaging sensor.

12. The vehicular control system of claim 1 , wherein the event point comprises an event that cannot be handled by the vehicular control system when the vehicular control system is operating to control driving of the vehicle.

13. The vehicular control system of claim 1 , wherein determination of the triggering event comprises detection of an object ahead of the vehicle and in a current path of travel of the vehicle.

14. The vehicular control system of claim 13 , wherein the event point comprises location of the detected object.

15. The vehicular control system of claim 13 , wherein the detected object comprises another vehicle ahead of the vehicle and in the current path of travel of the vehicle.

16. The vehicular control system of claim 13 , wherein the detected object comprises a pedestrian ahead of the vehicle and in the current path of travel of the vehicle.

17. The vehicular control system of claim 1 , comprising a plurality of radar sensors disposed at the vehicle and having fields of sensing exterior of the vehicle, and wherein the plurality of radar sensors includes the at least one forward-sensing sensor, and wherein the vehicular control system maneuvers the vehicle at least in part responsive to processing at the ECU of radar data captured by the plurality of radar sensors, and wherein the vehicular control system determines the triggering event responsive at least in part to processing at the ECU of the radar data captured by the plurality of radar sensors.

18. The vehicular control system of claim 1 , comprising a plurality of lidar sensors disposed at the vehicle and having fields of sensing exterior of the vehicle, and wherein the plurality of lidar sensors includes the at least one forward-sensing sensor, and wherein the vehicular control system maneuvers the vehicle at least in part responsive to processing at the ECU of lidar data captured by the plurality of lidar sensors, and wherein the vehicular control system determines the triggering event responsive at least in part to processing at the ECU of the lidar data captured by the plurality of lidar sensors.

19. The vehicular control system of claim 1 , wherein the in-cabin image data captured by the cabin-sensing imaging sensor is processed at the ECU.

20. A vehicular control system, the vehicular control system comprising:
a forward-viewing camera disposed at a windshield of a vehicle equipped with the vehicular control system, the forward-viewing camera viewing through the windshield forward of the vehicle, the forward-viewing camera capturing image data;
at least one forward-sensing sensor disposed at the vehicle and sensing forward of the vehicle, the at least one forward-sensing sensor capturing sensor data;
wherein t

Assignees

• Magna Electronics Inc

Inventors

• Latotzki Ralph
• Goseberg Frank

Classifications

• G05D1/2435

  Extracting 3D information · CPC title

• G05D1/81

  Handing over between on-board automatic and on-board manual control · CPC title

• G05D1/249

  from positioning sensors located off-board the vehicle, e.g. from cameras · CPC title

• B60W30/095

  Predicting travel path or likelihood of collision · CPC title

• B60G2400/823

  Obstacle sensing · CPC title