MikroTik RouterOS Watchdog: Automatic Recovery and System Monitoring
MikroTik RouterOS Watchdog: Automatic Recovery and System Monitoring
Section titled “MikroTik RouterOS Watchdog: Automatic Recovery and System Monitoring”RouterOS Version: 6.x / 7.x Difficulty: Beginner Estimated Time: 15 minutes
Overview
Section titled “Overview”The Watchdog feature provides automatic system recovery when your router becomes unresponsive or loses connectivity to a critical resource. It operates through two independent mechanisms:
- Software Watchdog Timer - Monitors system responsiveness and reboots if the system locks up for 60 seconds
- Ping Watchdog - Monitors connectivity to a specified IP address and reboots if unreachable
RouterBOARD devices also include a hardware watchdog that can recover the system even if the software watchdog fails - this operates at the hardware level and triggers if the OS becomes completely unresponsive.
Important: Watchdog reboots are intentional recovery actions, not errors. They indicate the system detected a problem and took corrective action.
Configuration
Section titled “Configuration”View Current Settings
Section titled “View Current Settings”/system watchdog printDefault Output:
watchdog-timer: yes watch-address: no-ping-delay: 5m ping-timeout: 60s automatic-supout: yes auto-send-supout: no send-email-to: send-email-from: send-smtp-server:Key Properties
Section titled “Key Properties”| Property | Default | Description |
|---|---|---|
watchdog-timer | yes | Enable software watchdog (reboots if system unresponsive for 60s) |
watch-address | (none) | IP address to ping; reboot if 6 consecutive pings fail |
no-ping-delay | 5m | Delay after boot before starting ping checks |
ping-timeout | 60s | Total time for ping check cycle (6 pings at 10s intervals) |
automatic-supout | yes | Generate support output file on crash |
auto-send-supout | no | Email support file after crash |
send-email-to | (none) | Recipient for support file |
send-email-from | (none) | Sender address for support file |
send-smtp-server | (none) | SMTP server for email delivery |
Configuration Steps
Section titled “Configuration Steps”Step 1: Enable Basic Watchdog (Default)
Section titled “Step 1: Enable Basic Watchdog (Default)”The software watchdog timer is enabled by default. Verify:
/system watchdog printExpected: watchdog-timer: yes
Step 2: Configure Ping Watchdog (Optional)
Section titled “Step 2: Configure Ping Watchdog (Optional)”Monitor a critical IP and reboot if unreachable:
/system watchdog set watch-address=192.168.1.1This pings the address every 10 seconds. After 6 consecutive failures (about 60 seconds), the router reboots.
Step 3: Set Ping Delay After Boot
Section titled “Step 3: Set Ping Delay After Boot”Prevent false reboots during startup when the network is still initializing:
/system watchdog set no-ping-delay=5mThis waits 5 minutes after boot before checking the watch-address.
Step 4: Configure Automatic Support Output (Recommended)
Section titled “Step 4: Configure Automatic Support Output (Recommended)”Enable automatic generation of diagnostic files when issues occur:
/system watchdog set automatic-supout=yesStep 5: Configure Email Notifications (Optional)
Section titled “Step 5: Configure Email Notifications (Optional)”Send support files via email after a crash:
/system watchdog set \ auto-send-supout=yes \ send-smtp-server=192.168.1.10Common Configuration Scenarios
Section titled “Common Configuration Scenarios”Scenario 1: Monitor Gateway Connectivity
Section titled “Scenario 1: Monitor Gateway Connectivity”Reboot if the default gateway becomes unreachable:
/system watchdog set watch-address=192.168.1.1 no-ping-delay=3mUse case: Remote site router that should reboot if uplink fails, potentially recovering from a stuck interface or routing issue.
Scenario 2: Monitor ISP DNS
Section titled “Scenario 2: Monitor ISP DNS”Reboot if unable to reach ISP’s DNS server:
/system watchdog set watch-address=8.8.8.8 no-ping-delay=5mCaution: Only use this if you’re certain the target is always reachable. Monitoring external addresses can cause unnecessary reboots if there are upstream issues beyond your control.
Scenario 3: Disable Ping Watchdog (Keep Software Watchdog)
Section titled “Scenario 3: Disable Ping Watchdog (Keep Software Watchdog)”/system watchdog set watch-address=""The software watchdog timer remains active, but no ping monitoring occurs.
Scenario 4: Completely Disable Watchdog
Section titled “Scenario 4: Completely Disable Watchdog”Not recommended, but if needed:
/system watchdog set watchdog-timer=no watch-address=""Warning: Without the watchdog, a system lockup requires manual intervention.
How the Ping Watchdog Works
Section titled “How the Ping Watchdog Works”- Router waits
no-ping-delaytime after boot (default: 5 minutes) - Sends ping to
watch-addressevery 10 seconds - Counts consecutive failures
- After 6 consecutive failures (~60 seconds), router reboots
- Cycle repeats after reboot
Critical point: With default settings, if the watch-address is unreachable, the router reboots approximately every 6 minutes (5 min delay + ~1 min ping timeout).
Verification
Section titled “Verification”Check 1: Verify Watchdog Configuration
Section titled “Check 1: Verify Watchdog Configuration”/system watchdog printExpected: Settings match your configuration.
Check 2: Check System Log for Watchdog Events
Section titled “Check 2: Check System Log for Watchdog Events”/log print where message~"watchdog"Expected: Shows any watchdog-related events.
Check 3: View Last Reboot Reason
Section titled “Check 3: View Last Reboot Reason”/system resource printLook for reboot-related information, or check:
/log print where topics~"system"Check 4: Verify Watch-Address is Reachable
Section titled “Check 4: Verify Watch-Address is Reachable”/ping 192.168.1.1 count=5Expected: Successful replies if using this as watch-address.
Troubleshooting
Section titled “Troubleshooting”Problem: “Router reboots every ~6 minutes”
Section titled “Problem: “Router reboots every ~6 minutes””Cause: Watch-address is unreachable, triggering continuous reboot cycle.
Solution:
- Immediately clear the watch-address:
/system watchdog set watch-address=""
- Verify connectivity to the intended watch-address:
/ping YOUR_WATCH_ADDRESS count=5
- If address is unreachable, fix the connectivity issue first
- Re-enable with a verified reachable address
Problem: “Watchdog reboots after RouterOS upgrade”
Section titled “Problem: “Watchdog reboots after RouterOS upgrade””Cause: RouterBoard firmware not upgraded after RouterOS upgrade.
Solution:
/system routerboard upgrade/system rebootAlways upgrade RouterBoard firmware after upgrading RouterOS.
Problem: “Random watchdog reboots with no clear pattern”
Section titled “Problem: “Random watchdog reboots with no clear pattern””Cause: Often power supply issues, especially on AC models.
Solution:
- Check power supply specifications match the device requirements
- Use the manufacturer’s power adapter (usually 24V for AC models)
- Avoid cheap DC-DC converters or regulators
- Check voltage with
/system health print(if available on your model) - Try a different power supply
Problem: “Watchdog reboot but watch-address is local and always up”
Section titled “Problem: “Watchdog reboot but watch-address is local and always up””Cause: The software watchdog timer triggered due to system unresponsiveness, not the ping watchdog.
Solution:
- Check for high CPU usage before the reboot
- Review logs for errors or issues
- Check for DDoS attacks or routing loops
- Generate and analyze supout.rif file
Problem: “No support file generated after crash”
Section titled “Problem: “No support file generated after crash””Cause: automatic-supout disabled or storage issue.
Solution:
/system watchdog set automatic-supout=yesVerify disk space:
/file printCommon Pitfalls
Section titled “Common Pitfalls”1. Monitoring an External IP That May Be Unreachable
Section titled “1. Monitoring an External IP That May Be Unreachable”Wrong:
/system watchdog set watch-address=8.8.8.8If your ISP has an outage, the router keeps rebooting - even though the router itself is fine.
Better: Monitor your gateway or a local device:
/system watchdog set watch-address=192.168.1.12. Setting Ping Delay Too Short
Section titled “2. Setting Ping Delay Too Short”Wrong:
/system watchdog set no-ping-delay=30sNetwork may not be fully initialized 30 seconds after boot, causing immediate reboot cycle.
Right:
/system watchdog set no-ping-delay=3mAllow time for interfaces, routing, and tunnels to come up.
3. Forgetting Watchdog is Enabled During Maintenance
Section titled “3. Forgetting Watchdog is Enabled During Maintenance”Problem: Router reboots during maintenance because monitored service is temporarily down.
Solution: Disable ping watchdog before maintenance:
/system watchdog set watch-address=""Re-enable after:
/system watchdog set watch-address=192.168.1.14. Expecting Watchdog to Fix All Problems
Section titled “4. Expecting Watchdog to Fix All Problems”Reality: Watchdog only helps with:
- System lockups (software watchdog)
- Specific connectivity failures (ping watchdog)
It won’t fix:
- Configuration errors
- Hardware failures
- Persistent software bugs
- Network issues beyond the router
5. Disabling Watchdog Completely
Section titled “5. Disabling Watchdog Completely”Wrong:
/system watchdog set watchdog-timer=noWithout the software watchdog, a system lockup requires physical access to recover.
Right: Keep software watchdog enabled; only adjust ping watchdog as needed.
Extended Connectivity Monitoring
Section titled “Extended Connectivity Monitoring”For more sophisticated monitoring than the built-in watchdog provides, use Netwatch:
# Monitor with custom timeout and actions/tool netwatch add host=8.8.8.8 interval=30s timeout=1s \ down-script="/log warning \"Gateway unreachable\"; /system reboot" \ comment="Custom watchdog"This provides:
- Configurable check intervals
- Longer grace periods before action
- Custom actions (not just reboot)
- Multiple monitored hosts
Hardware Watchdog
Section titled “Hardware Watchdog”RouterBOARD devices include a hardware watchdog that operates independently of RouterOS. If the operating system becomes completely unresponsive and can’t trigger the software watchdog, the hardware watchdog will still reboot the device.
This is automatic and requires no configuration - it’s a safety feature built into the hardware.
Limitations
Section titled “Limitations”- Ping watchdog only monitors one address
- Fixed 60-second software watchdog timeout (not configurable)
- No graduated response (always reboots, no warnings)
- Watchdog reboot doesn’t indicate specific cause in log
- Email notifications require working SMTP configuration
Related Features
Section titled “Related Features”- Netwatch (
/tool netwatch) - Advanced host monitoring with custom actions - Scheduler (
/system scheduler) - Time-based automated tasks - Scripts (
/system script) - Custom automation - Health (
/system health) - Hardware monitoring (voltage, temperature) - UPS (
/system ups) - Uninterruptible power supply monitoring
References
Section titled “References”Related Topics
Section titled “Related Topics”Extended Monitoring
Section titled “Extended Monitoring”- Netwatch - advanced host monitoring with custom actions
- Health Monitoring - hardware status monitoring (voltage, temperature)
Automation
Section titled “Automation”Notifications
Section titled “Notifications”- Email Tool - send support files and alerts via email
- SNMP - export monitoring data
Related Topics
Section titled “Related Topics”- System Backup - backup before potential watchdog issues
- Logging - review logs after watchdog events