Scuttle God

A god process monitor setup for the Secure Scuttlebutt Server (pub)

Things

• Notifications: webhook, email, twitter etc
• CPU and memory boundaries
• Lifecycle management with restarts, cool off periods and 'flapping' detection

Install

git clone ssb://%p2XzNsMYEWAVdZg0uIHJMp7JWoZe1ezcAc4IaeSYe08=.sha256 scuttle_god
cd scuttle_god
gem install god

Setup

# Setup accounts etc
vim sbot.god

Run

# detached
god -c sbot.god

# attached
god -c sbot.god -D

Omnipotence

I [2017-04-14 09:53:14]  INFO: Loading sbot.god
I [2017-04-14 09:53:14]  INFO: Syslog enabled.
I [2017-04-14 09:53:14]  INFO: Using pid file directory: /Users/joran/.god/pids
I [2017-04-14 09:53:14]  INFO: Started on drbunix:///tmp/god.17165.sock
I [2017-04-14 09:53:14]  INFO: Secure Scuttlebutt Server move 'unmonitored' to 'init'
I [2017-04-14 09:53:14]  INFO: Secure Scuttlebutt Server moved 'unmonitored' to 'init'
I [2017-04-14 09:53:14]  INFO: Secure Scuttlebutt Server [trigger] process is running (ProcessRunning)
I [2017-04-14 09:53:14]  INFO: Secure Scuttlebutt Server move 'init' to 'up'
I [2017-04-14 09:53:14]  INFO: Secure Scuttlebutt Server registered 'proc_exit' event for pid 93002
I [2017-04-14 09:53:14]  INFO: Secure Scuttlebutt Server moved 'init' to 'up'
I [2017-04-14 09:53:54]  INFO: Secure Scuttlebutt Server [trigger] memory out of bounds [*248244kb, *256632kb, *259632kb] (MemoryUsage)
I [2017-04-14 09:53:54]  INFO: Secure Scuttlebutt Server move 'up' to 'restart'
I [2017-04-14 09:53:54]  INFO: Secure Scuttlebutt Server deregistered 'proc_exit' event for pid 93002
I [2017-04-14 09:53:54]  INFO: Secure Scuttlebutt Server stop: default lambda killer
I [2017-04-14 09:53:54]  INFO: Secure Scuttlebutt Server sent SIGTERM
I [2017-04-14 09:53:55]  INFO: Secure Scuttlebutt Server process stopped
I [2017-04-14 09:53:55]  INFO: Secure Scuttlebutt Server start: sbot server
I [2017-04-14 09:53:55]  INFO: Secure Scuttlebutt Server moved 'up' to 'restart'
I [2017-04-14 09:53:55]  INFO: Secure Scuttlebutt Server [trigger] process is running (ProcessRunning)
I [2017-04-14 09:53:55]  INFO: Secure Scuttlebutt Server move 'restart' to 'up'
I [2017-04-14 09:53:55]  INFO: Secure Scuttlebutt Server registered 'proc_exit' event for pid 93035
I [2017-04-14 09:53:55]  INFO: Secure Scuttlebutt Server moved 'restart' to 'up'