EMG Watchdog is a perl script (emg_watchdog.pl) that is run as a background process, on the same server as EMG, which can monitor and control the emgd process. It is released with EMG 7 but can be used to control EMG 6 as well.
It implements a simple API based on HTTP and JSON for starting, stopping EMG and reading EMG server log files.
EMG Watchdog can be used from EMG Portal to control the EMG process via the web-based interface.
The script provided is named "emg_watchdog.pl.sample". It is intended to be copied to "emg_watchdog.pl" and can be run in the background using
as the same user that runs the "emgd" process, normally "emg".
It depends on several CPAN modules that need to be installed before the script can be run.
Before running the watchdog the configuration variables described below must be modified to match the local environment.
The latest version of the EMG Watchdog can be found on the URL below:
The most important variables below are "$emg_dir", "@notify_recipients" and "$mail_from".
Configuration variables in the script:
|$emg_dir||/home/emg/etc||Directory where EMG configuration (server.cfg) is located|
|$logfile||$emg_dir/log/emg_watchdog.log||Log file for EMG watchdog|
|$emg_quickstop||1||Use "quick stop" for stopping emgd (signal emgd to stop, wait a few seconds and then kill it)|
|$check_interval||60||Specifies how often (in seconds) checks should be run|
|$notify_interval||21600||Specifies how often (in seconds) notifications should be sent for persistent error (21600 secs = 6 hours)|
|$fs_to_check||/||File system to check for used space|
|$fs_limit||80||File system usage limit (in %), notification will be sent when limit exceeded|
|$listen_port||3000||Port on which integrated web server listens for incoming requests|
|@notify_recipients||('')||Array with recipients for e-mail notifications (empty by default)|
|$email@example.com||E-mail notification "From" address|
|$mail_subject_prefix||'EMG watchdog'||E-mail notification subject prefix (consider adding hostname for easier identification)|
|$smtp_server||127.0.0.1||IP address of SMTP server to use for sending e-mails|
|$smtp_port||25||Port of SMTP server to use for sending e-mails|
|$smtp_username||undef||Username for SMTP server authentication (undef = no auth)|
|$smtp_password||undef||Password for SMTP server authentication (undef = no auth)|
|$allowed_client_ips||127.0.0.1/32 192.168.0.0/24||Allowed client ip addresses / subnets for web server access|
When specific events occur the watchdog will send e-mail notifications to the addresses configured (@notify_recipients).
- Monitor detected EMG was not running
- EMG manual stop requested via API
- EMG stopped
- EMG manual startup requested via API
- EMG started
- EMG startup failed
- Used disk space in monitored file system ($fs_to_check) exceeds configured percent (fs_limit)
Using the watchdog API
The watchdog features a built-in web server which by default listens on port 3000 and implements an api.
The web server port must be protected in order to make sure it cannot be accessed directly via the internet. Access must be protected either by a local firewall in the server (iptables) or by an external firewall in front of the EMG server.
|URI||Description and parameters (M - Mandatory, O - Optional)|
|/api/ping||Check server status|
|/api/emg_start||Start EMG server|
|/api/emg_stop||Stop EMG server|
|/api/emg_status||Get status of EMG server|
|/api/get_log_file_list||Get list of log files|
|/api/get_log_file||Get contents of log file|
file - Log file name (M)
maxrows - Max number of rows to return (O, default: 100)
search_string - Only return rows that include search_string, multiple terms can be separated by space (O)
Sample http request using wget
should return the JSON response
Restarting the EMG Watchdog
EMG Watchdog is normally started on system boot via a line in /etc/rc.local:
This will run the watchdog as a background process owned by user "emg".
If the watchdog becomes unresponsive (EMG Portal says watchdog is not running even though process is running) it should be restarted by killing the process and starting it again.
You can do this when logged in as user "emg":