Watchdog....

[james]

I am sure a dozen solutions exist for this but I was bores so here is #13. The following software watches your spans (zaptel/dahdi) and if they go in to alarm it executes a hook script with 3 arguments:
arg 1 Span number
arg 2 Span Desription
arg 3 Alarm

You can write your hook script in any language and make it use said variables however you want. The original Idea was from a question about getting an email when the span went down, with this you could do a quick bash script to mail you when called by this program.

For Zaptel users:
Save the following code as watchdog.c in your zaptel directory. Then compile with `cc watchdog.c -o watchdog` directions for dahdi after the code.

//
//       (c)2010 James Finstrom <jfinsrom at gmail dot com>
//      Redistribution and use in source and binary forms, with or without
//      modification, are permitted provided that the following conditions are
//      met:
//      
//      * Redistributions of source code must retain the above copyright
//        notice, this list of conditions and the following disclaimer.
//      * Redistributions in binary form must reproduce the above
//        copyright notice, this list of conditions and the following disclaimer
//        in the documentation and/or other materials provided with the
//        distribution.
//      * Neither the name of James Finsrom nor the names of its
//        contributors may be used to endorse or promote products derived from
//        this software without specific prior written permission.
//      
//      THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
//      "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
//      LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
//      A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
//      OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
//      SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
//      LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
//      DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
//      THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
//      (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
//      OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
//
//      based on ztscan.c by Brandon Kruse <[email protected]> 
//      and Kevin P. Fleming <[email protected]>
//
#include <stdio.h> 
#include <string.h>
#include <stdarg.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/ioctl.h>
#include <fcntl.h>
#include <errno.h>

#ifdef STANDALONE_ZAPATA
#include "kernel/zaptel.h"
#else
#include <zaptel/zaptel.h>
#endif

int main(int argc, char *argv[])
{
	int ctl;
	int x, y;
	struct zt_params params;
	unsigned int basechan = 1;
	struct zt_spaninfo s;
	char buf[100];
	char buf2[100];
	char alarms[50];
   	char scriptname[100];
	if( argc < 2 ) {
		printf("USAGE: watchdog hook_script_name\n");
		exit(1);
	}
	
   	strcpy(scriptname, argv[1]);

	if ((ctl = open("/dev/zap/ctl", O_RDWR)) < 0) {
		fprintf(stderr, "Unable to open /dev/zap/ctl: %s\n", strerror(errno));
		exit(1);
	}
while(1){	
	for (x = 1; x < ZT_MAX_SPANS; x++) {
		memset(&s, 0, sizeof(s));
		s.spanno = x;
		if (ioctl(ctl, ZT_SPANSTAT, &s))
			continue;

		alarms[0] = '\0';
		if (s.alarms) {
			if (s.alarms & ZT_ALARM_BLUE)
				strcat(alarms,"BLU/");
			if (s.alarms & ZT_ALARM_YELLOW)
				strcat(alarms, "YEL/");
			if (s.alarms & ZT_ALARM_RED)
				strcat(alarms, "RED/");
			if (s.alarms & ZT_ALARM_LOOPBACK)
				strcat(alarms,"LB/");
			if (s.alarms & ZT_ALARM_RECOVER)
				strcat(alarms,"REC/");
			if (s.alarms & ZT_ALARM_NOTOPEN)
				strcat(alarms, "NOP/");
			if (!strlen(alarms))
				strcat(alarms, "UUU/");
			if (strlen(alarms)) {
				/* Strip trailing / */
				alarms[strlen(alarms)-1]='\0';
			}
		} else {
			if (s.numchans)
				strcpy(alarms, "OK");
			else
				strcpy(alarms, "UNCONFIGURED");
		}
                if(strcmp(alarms, "OK") == 0 || strcmp(alarms, "UNCONFIGURED") == 0){
	        //LIFE IS GOOD
		sleep(1);
		continue;
		}else{
                sprintf(buf2, "'%s' 'SPAN: %i' '%s' '%s'",scriptname, x, s.desc, alarms);
                system(buf2);
                sleep(500);
}
		basechan += s.totalchans;
	}
    }
	exit(0);
}

For DAHDI users:
Save the following code as watchdogdahdi.c in your dahdi/tools/ directory.
Then compile with `cc watchdogdahdi.c -o dahdi_watchdog`

//
//       (c)2010 James Finstrom <jfinsrom at gmail dot com>
//      Redistribution and use in source and binary forms, with or without
//      modification, are permitted provided that the following conditions are
//      met:
//      
//      * Redistributions of source code must retain the above copyright
//        notice, this list of conditions and the following disclaimer.
//      * Redistributions in binary form must reproduce the above
//        copyright notice, this list of conditions and the following disclaimer
//        in the documentation and/or other materials provided with the
//        distribution.
//      * Neither the name of James Finsrom nor the names of its
//        contributors may be used to endorse or promote products derived from
//        this software without specific prior written permission.
//      
//      THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
//      "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
//      LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
//      A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
//      OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
//      SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
//      LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
//      DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
//      THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
//      (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
//      OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
//
//      based on DAHDIscan.c by Brandon Kruse <[email protected]> 
//      and Kevin P. Fleming <[email protected]>
//
#include <stdio.h> 
#include <string.h>
#include <stdarg.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/ioctl.h>
#include <fcntl.h>
#include <errno.h>

#ifdef STANDALONE_ZAPATA
#include "kernel/user.h"
#else
#include <dahdi/user.h>
#endif

int main(int argc, char *argv[])
{
	int ctl;
	int x, y;
	struct dahdi_params params;
	unsigned int basechan = 1;
	struct dahdi_spaninfo s;
	char buf[100];
	char buf2[100];
	char alarms[50];
   	char scriptname[100];
	if( argc < 2 ) {
		printf("USAGE: dahdi_watchdog hook_script_name\n");
		exit(1);
	}
	
   	strcpy(scriptname, argv[1]);

	if ((ctl = open("/dev/dahdi/ctl", O_RDWR)) < 0) {
		fprintf(stderr, "Unable to open /dev/dahdi/ctl: %s\n", strerror(errno));
		exit(1);
	}
while(1){	
	for (x = 1; x < DAHDI_MAX_SPANS; x++) {
		memset(&s, 0, sizeof(s));
		s.spanno = x;
		if (ioctl(ctl, DAHDI_SPANSTAT, &s))
			continue;

		alarms[0] = '\0';
		if (s.alarms) {
			if (s.alarms & DAHDI_ALARM_BLUE)
				strcat(alarms,"BLU/");
			if (s.alarms & DAHDI_ALARM_YELLOW)
				strcat(alarms, "YEL/");
			if (s.alarms & DAHDI_ALARM_RED)
				strcat(alarms, "RED/");
			if (s.alarms & DAHDI_ALARM_LOOPBACK)
				strcat(alarms,"LB/");
			if (s.alarms & DAHDI_ALARM_RECOVER)
				strcat(alarms,"REC/");
			if (s.alarms & DAHDI_ALARM_NOTOPEN)
				strcat(alarms, "NOP/");
			if (!strlen(alarms))
				strcat(alarms, "UUU/");
			if (strlen(alarms)) {
				/* Strip trailing / */
				alarms[strlen(alarms)-1]='\0';
			}
		} else {
			if (s.numchans)
				strcpy(alarms, "OK");
			else
				strcpy(alarms, "UNCONFIGURED");
		}
                if(strcmp(alarms, "OK") == 0 || strcmp(alarms, "UNCONFIGURED") == 0){
	        //LIFE IS GOOD
		sleep(1);
		continue;
		}else{
                sprintf(buf2, "'%s' 'SPAN %i' '%s' '%s'",scriptname, x, s.desc, alarms);
                system(buf2);
                sleep(500);
}
		basechan += s.totalchans;
	}
    }
	exit(0);
}

Zaptel:
Usage: watchdog hook_script.lang
DAHDI:
Usage: dahdi_watchdog hook_script.lang

You probably want to put this in your sbin folder...

Zaptel: install -m 755 watchdog /usr/local/sbin
DAHDI: install -m 755 dahdi_watchdog /usr/local/sbin

 

[jroper]

Hi

Nice idea. here's a possiblility for solution #14

OSSEC, and I believe Fail2Ban, as log parsers, can be used in a similar fashion by looking out for strings that indicate problems in the logs, and sending an email.

Joe

 

[james]

Joe, I can see where that would be actually better, this depends on alarms which alerts you of layer 1 and 2 stuff but the D channel can go down without the span alarming. I waned to play with C and thought I would share my toys but a log parser would probably be better suited, that said maybe that will be next.

 

[james]

Added a discussion on doing the fail2ban thing http://pbxinaflash.com/community/threads/fail2ban-as-a-pri-monitor.6930/#post-41895\