Simon icon Simon
Flexible server monitoring

Question or feature request: do task after x multiple failures

I may have overlooked this feature, but if not...

David, is there a way of having Simon do nothing on the first or second 'failure', but after 3 in a row, it would then notify/perform a task?

As I demo this product, I am very happy with it. Except for occasional false positives that occur (primarily with a POP server test, but on occasion with a heavily-loaded http system).

In Whistleblower, the notifiers were set up in a manner that each 'failure' could be assigned a notification/task. E.g.,

On the first failure, beep
On the second failure, speak a warning
On the third failure, send an email
On the fourth failure, run a script (to reset a PDU or something similar).

Each 'failure' would have a timeout period, and a lag period in between. If any stage tested OK, then the failure count would be returned to zero.

Presently, I have Simon beep and send an email upon an individual failure. I'm concerned that if I try to implement the APC scripts, or craft my own, the false positive rate would be too high for the health of the server, causing undue resets.

Thoughts on this? Do you see my dilemma? Simon's a beautiful product and will easily replace the MIA Whistleblower in our shop. It is this one last function that has me concerned.

Thank you.

David Sinclair's picture

Re: do task after x multiple failures

Simon does support only performing notifications after a number of consecutive failures. See the Edit Notifier ▶ Options section. For example, you can configure a notifier to only notify if two consecutive failures have occurred, among other options.

You can also assign multiple notifiers for a failure, and configure them as desired, to do a similar setup to your old one. Have a Sound notifier perform after 1 failure, a Speech notifier perform after 2 failures, an E-mail notifier perform after 3 failures, and a Script notifier perform after 4 failures. It works in multiples, so the Sound would perform every time (unless you tell it not to perform after the first failure), Speech would be every second consecutive failure, and so on — but that'd be quite useful.

Ah...

dude... that's brilliant. Exactly what I was looking for, now that I understand the interface a bit better.

Wow. Now I'm going to have to send you money.

David Sinclair's picture

Re: Ah...

Music to my ears. :)

If you have any ideas on how the interface could be made more intuitive, please let me know. Feedback is always most welcome.