System fault detection and processing method, device, and computer readable storage medium

Patent metadata

Field	Value
Publication number	US-9720761-B2
Application number	US-201414781403-A
Country	US
Kind code	B2
Filing date	Jan 6, 2014
Priority date	Apr 1, 2013
Publication date	Aug 1, 2017
Grant date	Aug 1, 2017

Abstract

Disclosed are a method, a device, and a computer readable storage medium for detecting and processing a system fault. The method includes: an interrupt service routine sending a first stage kicking dog signal, and receiving a second stage kicking dog signal for a system detection task (S 101 ); and when task dead loop or task abnormity is detected, performing system abnormity processing according to a preset processing policy, wherein when the interrupt service routine fails to receive the second stage kicking dog signal within a set period of time, the interrupt service routine stops sending the first stage kicking dog signal, and the system reboots (S 102 ).

First claim

What is claimed is:

1. A method for detecting and processing a system fault, comprising:
sending, by an interrupt service routine, a first stage kicking dog signal, and receiving a second stage kicking dog signal for a system detection task; and
when task dead loop or task abnormity is detected, performing system abnormity processing according to a preset processing policy,
wherein when the interrupt service routine fails to receive the second stage kicking dog signal within a set period of time, the interrupt service routine stops sending the first stage kicking dog signal, and the system reboots.

2. The method according to claim 1 , further comprising: when operating system breakdown or hardware abnormity occurs in the system, automatically rebooting, by the system, for recovery.

3. The method according to claim 1 , wherein when an interrupt exceeds a set threshold, a task with a higher priority over the system detection task is busy, system abnormity occurs during startup of the system or the system detection task suspends due to abnormity in itself, the interrupt service routine fails to receive the second stage kicking dog signal.

4. The method according to claim 3 , wherein the detection of the task dead loop comprises:
timing, by the system detection task, a second stage software kicking dog, and timing, by a low priority dead loop auxiliary task, a keep-alive maintaining for the dead loop;
calculating CPU occupation ratio periodically;
determining whether the calculated CPU occupation ratio is higher than a CPU dead loop determining threshold; in the case that the calculated CPU occupation ratio is not higher than the CPU dead loop determining threshold, determining that no task dead loop occurs; in the case that the calculated CPU occupation ratio is higher than the CPU dead loop determining threshold, determining whether the low priority dead loop auxiliary task is set to be keep-alive; in the case that the low priority dead loop auxiliary task is set to be keep-alive, determining that no task dead loop occurs; otherwise, alarming and notifying maintenance personnel for analysis; and
determining whether the system detection task handled only one message during a sample detection period; in the case that the system detection task handled more than one message during the sample detection period, alarming and notifying maintenance personnel for analysis; otherwise, determining that the task dead loop occurs.

5. The method according to claim 1 , wherein the detection of the task abnormity comprises:
detecting working state of all tasks periodically; and
performing the task abnormity detection according to the detected working state in conjunction with a pre-configured task abnormity determination policy.

6. The method according to claim 2 , wherein when an interrupt exceeds a set threshold, a task with a higher priority over the system detection task is busy, system abnormity occurs during startup of the system or the system detection task suspends due to abnormity in itself, the interrupt service routine fails to receive the second stage kicking dog signal.

7. The method according to claim 2 , wherein detecting the task abnormity comprises:
detecting working state of all tasks periodically; and
performing the task abnormity detection according to the detected working state in conjunction with a pre-configured task abnormity determination policy.

8. The method according to claim 4 , wherein detecting the task abnormity comprises:
detecting working state of all tasks periodically; and
performing the task abnormity detection according to the detected working state in conjunction with a pre-configured task abnormity determination policy.

9. A device for detecting and processing a system fault, comprising:
a signal processing module configured to cause an interrupt service routine to send a first stage kicking dog signal, and receive a second stage kicking dog signal for a system detection task; and
an abnormity processing module configured to, when task dead loop or task abnormity is detected, perform system abnormity processing according to a preset processing policy, wherein, when the interrupt service routine fails to receive the second stage kicking dog signal within a set period of time, the abnormity processing module causes the interrupt service routine to stop sending the first stage kicking dog signal and causes the system to reboot.

10. The device according to claim 9 , further comprising:
an automatic reboot module configured to, when operating system breakdown or hardware abnormity occurs in the system, reboot the system automatically for recovery.

11. The device according to claim 9 , wherein when an interrupt exceeds a set threshold, a task with a higher priority over the system detection task is busy, system abnormity occurs during startup of the system or the system detection task suspends due to abnormity in itself, the interrupt service routine fails to receive the second stage kicking dog signal.

12. The device according to claim 11 , further comprising:
a CPU occupation ratio calculation module configured to, when the system detection task times a second stage software kicking dog, and a low priority dead loop auxiliary task times keep-alive maintaining for the dead loop, calculate CPU occupation ratio periodically; and
a task dead loop detection module configured to, in the case that the system detection task determines that the calculated CPU occupation ratio is not higher than a CPU dead loop determining threshold, determine that no task dead loop occurs; in the case that the system detection task determines that the calculated CPU occupation ratio is higher than the CPU dead loop determining threshold, determine whether the low priority dead loop auxiliary task is set to be keep-alive; in the case that the low priority dead loop auxiliary task is set to be keep-alive, determine that no task dead loop occurs; otherwise, alarm and notify maintenance personnel for analysis; and
wherein the task dead loop detection module is further configured to determine whether the system detection task handled only one message during a sample detection period; in the case that the system detection task handled more than one message during the sample detection period, alarm and notify maintenance personnel for analysis; otherwise, determine that the task dead loop occurs.

13. The device according to claim 9 , further comprising:
a task working state detection module configured to detect working state of all tasks periodically; and
a task abnormity detection module configured to perform the task abnormity detection according to the detected working state in conjunction with a pre-configured task abnormity determination policy.

14. The device according to claim 10 , wherein when an interrupt exceeds a set threshold, a task with a higher priority over the system detection task is busy, system abnormity occurs during startup of the system or the system detection task suspends due to abnormity in itself, the interrupt service routine fails to receive the second stage kicking dog signal.

15. The device according to claim 10 , further comprising:
a task working state detection module configured to detect working state of all tasks periodically; and
a task abnormity detection module configured to perform the task abnormity detection according to the detected working state in conjunction with a pre-configured task abnormity determination policy.

16. The device according to claim 12 , further comprising:
a task working state detection module configured to detect working s

Assignees

• Zte Corp

Inventors

• Yu Guangbo
• Zhu Huaiyun
• Qiu Jing

Classifications

• G06F11/0715 Primary

  in a system implementing multitasking (multitasking per se G06F9/46) · CPC title

• G06F11/0751

  Error or fault detection not based on redundancy (power supply failures G06F1/30; network fault management H04L41/06) · CPC title

• G06F11/0757

  by exceeding a time limit, i.e. time-out, e.g. watchdogs · CPC title

• G06F2201/85

  Active fault masking without idle spares · CPC title

• G06F11/079 Primary

  Root cause analysis, i.e. error or fault diagnosis (in a hardware test environment G06F11/22; in a software test environment G06F11/36) · CPC title