Fatigue detection and operator alert system

Patent metadata

Field	Value
Publication number	US-12539870-B2
Application number	US-202318209352-A
Country	US
Kind code	B2
Filing date	Jun 13, 2023
Priority date	Jun 13, 2023
Publication date	Feb 3, 2026
Grant date	Feb 3, 2026

Abstract

A fatigue detection and operator alert system is disclosed for detecting operator fatigue in a machine having a cab, a seat, and a joystick. The fatigue detection and operator alert system comprises an optic sensor positioned non-obstructively facing an operator and configured to output a first signal consisting of images captured of an operator; a pressure sensor configured to provide a second signal indicative of a pressure of an operator hand on the joystick; a thermal sensor configured to provide a third signal indicative of a temperature of the operator; a load sensor configured to provide a fourth signal indicative of a load of an operator foot; and a controller configured to: receive the sensor signals; detect instances of operator fatigue based on the received sensor signals during non-hauling operation; and actuate an alert device to alert the operator when instances of operator fatigue are detected.

First claim

What is claimed is:

1 . A fatigue detection and operator alert system for a machine having a cab, a seat, and a joystick, comprising:
an optic sensor positioned non-obstructively facing an operator without interfering with a viewing area of the operator, and the optic sensor is configured to output a first signal consisting of images captured of the operator; a joystick sensor assembly operatively associated with the joystick and including: a pressure sensor configured to provide a second signal indicative of a pressure of an operator hand on the joystick, and a thermal sensor configured to provide a third signal indicative of a temperature of the operator; and a controller in communication with the optic sensor and the joystick sensor assembly, the controller configured to:
receive the first signal, the second signal, and the third signal;
detect instances of operator fatigue in real time based on the first signal, the second signal, and the third signal during non-hauling operation of the machine; and
actuate an alert device communicably coupled to the controller, the alert device configured to receive an actuation signal and issue an operator alert when instances of operator fatigue are detected.

2 . The fatigue detection and operator alert system of claim 1 , further comprising a load sensor configured to provide a fourth signal indicative of a weight of an at least one operator foot, the controller configured to receive the fourth signal.

3 . The fatigue detection and operator alert system of claim 2 , wherein:
the first signal received by the controller from the optic sensor includes images captured of operator head movements; the second signal received by the controller from the pressure sensor includes a grip pressure of the operator hand or fingers; the third signal received by the controller from the thermal sensor includes the temperature associated with a hand or at least one finger on the hand of the operator; and the fourth signal received by the controller from the load sensor includes a measurement of weight of the operator on the load sensor.

4 . The fatigue detection and operator alert system of claim 3 , the controller is disposed in communication an off-board network, and a back-office system, wherein the alert device includes at least one of:
an audible alarm; a seat vibration; a visual alarm on a display panel; a lighting alarm; and a back-office system notification of instances of operator fatigue.

5 . The fatigue detection and operator alert system of claim 3 , wherein:
the optic sensor is at least one of an infrared sensor, an optical camera, a stereo camera, a smart camera, a monocular camera, a wired ethernet cameras with high compute PGA, and a smart vision system; the pressure sensor is at least one of a strain gauge sensor, a piezoelectric sensor, a capacitive pressure sensor, a resistive sensor, and a Hall effect sensor; the thermal sensor is at least one of a thermocouple, a thermistor, a resistance temperature detector, and an infrared temperature sensor; and the load sensor is at least one of an electro-mechanical sensor, a load cell sensor, strain gauge load cells, hydraulic load cells, pneumatic load cells, capacitive load cells, piezoelectric load cells, and a magnetic load cells.

6 . The fatigue detection and operator alert system of claim 5 , wherein the controller is configured to process the images captured of operator head movements received from the optic sensor to detect instances of operator fatigue when operator head movements occur relative to at least one of:
the seat fixed in the cab; the cab fixed on a frame of the machine; movements by a rotatable seat in the cab; movements by a rotatable floorboard in the cab; movements by a rotatable cab on the frame of the machine; and movements by an upper swiveling body on the machine.

7 . The fatigue detection and operator alert system of claim 6 , wherein the controller is configured with a machine learning and pattern recognition model to detect instances of operator fatigue by interpreting patterns and features based on the received first, second, third, and fourth signals.

8 . The fatigue detection and operator alert system of claim 7 , wherein the joystick sensor assembly receives touch and grip inputs from the operator including:
a finger pressure; a hand grip pressure; a finger temperature; a hand temperature; a joystick button actuation; and a joystick movement.

9 . A machine comprising:
a frame; one or more prime movers mounted on the frame; a work implement moveably mounted on the frame and operatively driven by at least one of the one or more prime movers; a cab; a seat mounted on a floorboard of the cab; a joystick for actuating the work implement; a fatigue detection and operator alert system including:
an optic sensor positioned behind an operator and non-obstructively facing the operator without interfering with a viewing area, and the optic sensor is configured to output a first signal consisting of images captured of the operator;
a joystick sensor assembly operatively associated with the joystick and including: a pressure sensor configured to provide a second signal indicative of a pressure of an operator hand on the joystick, and a thermal sensor configured to provide a third signal indicative of a temperature of the operator;
a load sensor configured to provide a fourth signal indicative of a weight of the operator;
a controller in communication with the optic sensor, the joystick sensor assembly, and the load sensor, the controller configured to:
receive the first signal, the second signal, the third signal, and the fourth signal;
detect instances of operator fatigue in real time based on the first signal, the second signal, the third signal and the fourth signal during non-hauling operation of the machine; and
actuate an alert device communicably coupled to the controller, the alert device configured to receive an actuation signal and issue an operator alert when instances of operator fatigue are detected.

10 . The machine of claim 9 , wherein:
the first signal received by the controller from the optic sensor includes images captured of operator head movements; the second signal received by the controller from the pressure sensor includes a grip pressure of the operator hand or fingers; the third signal received by the controller from the thermal sensor includes the temperature associated with a hand or at least one finger on the hand of the operator; and the fourth signal received by the controller from the load sensor includes a measurement of weight of the operator on the load sensor.

11 . The machine of claim 10 , the controller is disposed in communication with an off-board network, a remote, and a back-office system.

12 . The machine of claim 11 , wherein the alert device includes at least one of:
an audible alarm; a seat vibrator; a visual alarm on a display panel; a lighting alarm; and a back-office system notification of instances of operator fatigue.

13 . The machine of claim 12 , wherein the controller is configured to process the images captured of operator head movements received from the optic sensor to detect instances of operator fatigue when operator head movements occur relative to at least one of:
the seat fixed in the cab; the cab fixed on the frame of the machine; movements by a rotatable seat in the cab; movements by a rotatable floorboard in the cab; movements by a rotatable cab on the frame of the machine; and movements by an upper swiveling body on the machine.

14 . The machi

Assignees

• Caterpillar Inc

Inventors

• Kumar Luvkush
• Mukherjee Tarashankar
• Hamann Robert A
• Labre Jeffrey P

Classifications

• B60W2540/26

  Incapacity · CPC title

• B60W2540/229

  Attention level, e.g. attentive to driving, reading or sleeping · CPC title

• B60W2540/221

  Physiology, e.g. weight, heartbeat, health or special needs · CPC title

• B60W2420/403

  Image sensing, e.g. optical camera · CPC title

• G06V20/597

  Recognising the driver's state or behaviour, e.g. attention or drowsiness · CPC title