Sensor fusion for autonomous driving transition control

Patent metadata

Field	Value
Publication number	US-10481602-B2
Application number	US-201615295039-A
Country	US
Kind code	B2
Filing date	Oct 17, 2016
Priority date	Oct 17, 2016
Publication date	Nov 19, 2019
Grant date	Nov 19, 2019

Abstract

A system for sensor fusion for autonomous driving transition control includes a sensor fusion module and a decision making module. The sensor fusion module fuses a plurality of steering sensor data from one or more sensors of a steering system with a plurality of driver state data from a plurality of driver state sensors as a plurality of fused state data aligned in time. The decision making module determines whether to transition from an autonomous driving mode to a manual driving mode based on the fused state data.

First claim

The invention claimed is:

1. A system for sensor fusion for autonomous driving transition control, the system comprising:
a processor operable to receive a plurality of steering sensor data from one or more sensors of a steering system and receive a plurality of driver state data from a plurality of driver state sensors, the processor configured to:
fuse the steering sensor data with the driver state data as a plurality of fused state data aligned in time;
determine whether to transition from an autonomous driving mode to a manual driving mode based on the fused state data;
command a steering motor actuator of the steering system autonomously in the autonomous driving mode;
command the steering motor actuator responsive to a driver input in the manual driving mode;
determine an attentive state of a driver based on time-averaging of the fused state data;
continue monitoring the fused state data after the transition from the autonomous driving mode to the manual driving mode has been initiated to determine an attentive state of a driver based on time-averaging of the fused state data; and
revert from the manual driving mode to the autonomous driving mode based on determining that the attentive state of the driver no longer meets one or more transition validity criteria.

2. The system of claim 1 , wherein the steering sensor data comprise sensed or derived data from one or more of a handwheel torque sensor and a handwheel position/velocity sensor.

3. The system of claim 1 , wherein the driver state data comprise sensed or derived data from one or more of handwheel touch sensor and a camera.

4. The system of claim 3 , wherein the driver state data sensed or derived from the handwheel touch sensor comprises one or more of: an area of contact with a handwheel, a width of contact with the handwheel, a force of contact with the handwheel, and a position of contact with the handwheel, and wherein the driver state data sensed or derived from the camera comprises one or more of: a driver body posture, a driver head pose, a driver eye gaze, and a driver hand position.

5. The system of claim 1 , further comprising a recent data store of the fused state data and a history data store of the fused state data, wherein the history data store is operable to collect older values of the fused state data than collected in the recent data store.

6. The system of claim 5 , wherein the processor is configured to average multiple samples of the fused state data from the history data store as time-averaged fused state data and determine whether to transition from the autonomous driving mode to the manual driving mode based on the fused state data from the recent data store in combination with the time-averaged fused state data.

7. The system of claim 1 , wherein the processor is further configured to revert from the manual driving mode to the autonomous driving mode based on determining that an input from one or more of the driver state sensors is obstructed while monitoring the fused state data after the transition from the autonomous driving mode to the manual driving mode has been initiated.

8. A steering system comprising:
one or more sensors operable to produce a plurality of steering sensor data;
a plurality of driver state sensors operable to produce a plurality of driver state data;
a steering actuator motor; and
a control module comprising a processor operable to command the steering actuator motor in an autonomous driving mode and in a manual driving mode, the control module further operable to fuse the steering sensor data with the driver state data as a plurality of fused state data aligned in time, determine whether to transition from the autonomous driving mode to the manual driving mode based on the fused state data, determine an attentive state of a driver based on time-averaging of the fused state data, continue monitoring the fused state data after the transition from the autonomous driving mode to the manual driving mode has been initiated to determine an attentive state of a driver based on time-averaging of the fused state data, and revert from the manual driving mode to the autonomous driving mode based on determining that the attentive state of the driver no longer meets one or more transition validity criteria.

9. The steering system of claim 8 , wherein the steering sensor data comprise sensed or derived data from one or more of a handwheel torque sensor and a handwheel position/velocity sensor.

10. The steering system of claim 8 , wherein the driver state data comprise sensed or derived data from one or more of handwheel touch sensor and a camera.

11. The steering system of claim 10 , wherein the driver state data sensed or derived from the handwheel touch sensor comprises one or more of: an area of contact with a handwheel, a width of contact with the handwheel, a force of contact with the handwheel, and a position of contact with the handwheel, and wherein the driver state data sensed or derived from the camera comprises one or more of: a driver body posture, a driver head pose, a driver eye gaze, and a driver hand position.

12. The steering system of claim 8 , wherein the control module comprises a recent data store of the fused state data and a history data store of the fused state data, wherein the history data store is operable to collect older values of the fused state data than collected in the recent data store.

13. The steering system of claim 12 , wherein the control module is operable to average multiple samples of the fused state data from the history data store as time-averaged fused state data and determine whether to transition from the autonomous driving mode to the manual driving mode based on the fused state data from the recent data store in combination with the time-averaged fused state data.

14. The steering system of claim 8 , wherein the processor is further configured to revert from the manual driving mode to the autonomous driving mode based on determining that an input from one or more of the driver state sensors is obstructed while monitoring the fused state data after the transition from the autonomous driving mode to the manual driving mode has been initiated.

15. A method for sensor fusion for autonomous driving transition control, the method comprising:
acquiring, by a processor of a control module, a plurality of steering sensor data from one or more sensors of a steering system;
acquiring, by the processor of the control module, a plurality of driver state data from a plurality of driver state sensors;
fusing, by the processor of the control module, the steering sensor data with the driver state data as a plurality of fused state data aligned in time;
determining, by the processor of the control module, whether to transition from an autonomous driving mode to a manual driving mode based on the fused state data;
commanding, by the processor of the control module, a steering motor actuator of the steering system autonomously in the autonomous driving mode;
commanding, by the processor of the control module, the steering motor actuator responsive to a driver input in the manual driving mode;
determining, by the processor of the control module, an attentive state of a driver based on time-averaging of the fused state data;
continuing monitoring, by the processor of the control module, the fused state data after the transition from the autonomous driving mode to the manual driving mode has been initiated to determine an attentive state of a driver based on time-averaging of the fused state data; and
reverting, by the processor of the control module, from the manual driving mode

Assignees

• Steering Solutions Ip Holding

Inventors

• Chandy Ashok

Classifications

• B60W2510/202

  Steering torque · CPC title

• B60W2510/20

  Steering systems · CPC title

• B60W2510/205

  Steering speed · CPC title

• B60W30/10

  Path keeping {(cruise control for automatically following a preceding vehicle B60W30/165)} · CPC title

• B62D1/286

  Systems for interrupting non-mechanical steering due to driver intervention · CPC title