Asterisk monitoring
Now that you've built Asterisk around your entire life it might be a good idea to make sure it stays up!
Note: This doesn't appear to be supported in argus-3.3
We at ITConnection.ru are happy to post a Cacti script for Asterisk IP-PBX statistics. It's based on Python, works over Asterisk manager API connection and provides cute and comprehensive graphs. You can download Cacti's plug-in from this page: http://forums.cacti.net/viewtopic.php?p=111317
A simple app collecting this data should be easy to write
1. Installing a basic SNMP agent on each Linux box and using a central SNMP manager to monitor each node. This would give notice when a remote node became isolated from the monitoring network.
2. Rolling in Asterisk alarm logs into a syslog server or even as SNMP traps.
3. Writing a small app that simply interrogates those interfaces that are important to the operation (iax2/udp, sip/udp, etc, send a crafted pkt and interpret the returned result. Port not open is obvious, no response is obvious, incorrect response is not so obvious)
4. Test call to an outside number once per five minutes, hourly, or whatever trips your trigger (outside number only needs to respond with something that is predictable, doesn't have to be a person or company)
5. Monitoring logs looking for keywords (may take some time to identify the appropriate keywords) (e.g. Swatch or Logwatch)
It would seem logical for someone to write a res_snmp.c for asterisk that would expose a lot of asterisk's internal data. This would seem a logical step toward writing fully functional monitoring applications as well. The module would allow clients to add themselves to the list and receive traps, as well as check for the current status of various variables.
_Idea is ours and now need to write code....... please include code here ....... Nahid Hossain_
James Thompson: I'd suggest using something like Sipsak to actually do a SIP registration to verify the service is up.
You could use the Whats Up Gold send/expect to connect to Linux box, run sipsak, and then check the results.
<brainstorming>
Okay, this may be over the top, but here goes. Write an asterisk application that sends (and receives) status information to another box over the PSTN. My idea is not only to use this as a way to verify that * is running, but asa way to RELIABLY tell that a remote * box is actively accepting incoming calls. It wouldn't have to be anything complicated, just a heartbeat and some basic details to let the caller know that "yes, I'm alive and acceptingcalls over this line". Simplified protocol:
1) Monitoring box calls up and says (in DTMF):
#<my CallerID>
#<extension I am trying to reach>
#<I'm a machine, so reply in DTMF instead of voice>
#<the secret code is>#
2) The remote box says:
#<your CallerID>#<Your DNIS>
#<yes I will accept a call to thatnumber>#
3) Monitoring box acknowledges and disconnects
4) Remote box disconnects
5) Monitoring box decides whether it likes the answers it received and performs actions accordingly.
</brainstorming>
Hint: Do a "show application AlarmReceiver" and you are very close to what you brainstormed... I think...
TJH- (nicheware)
I've had some problems with peers losing their registrations, usually due to crappy ISP's with DNS problems.
This script tests if peers are up. (registered) I call it from a cron job. If a peer loses it's registration I receive an e-mail.
#!/bin/bash
for peername in peer_1 peer_2 peer_n
do
# note, next lines are "command substitution" - use backticks ` not quotations ' as appropriate
otpt=`/usr/sbin/asterisk -rx 'iax2 show peers' |grep $peername |grep Unspecified`
if [ -n "$otpt" ]; then
echo $otpt | mail -s "Asterisk peer: \"`echo -n $peername `\" has a problem." "you@yourdomain.com"
fi
done
down. Sometimes if their asterisk is just hosed they will still have the same IP- so I can ssh in and fix it. Also it would be
nice to send either an electronic page or even a call file to alert me of problems - although I don't want the phone ringing
at 4AM.
-TJH
HAL
grep has problems with not printable chars, so you should use "strings -a" in your script.
otpt=`/usr/sbin/asterisk -rx 'iax2 show peers' |strings -a| grep $peername |grep Unspecified`
HAL
Another note from a user on a large scale (S McGowan, 2005)
We're using Zabbix to run local UserParameter commands to monitor Asterisk. We use asterisk -rx commands to output what we want and then return the result to the Zabbix monitoring server. There's been small problems with too many remote connections at once crashing asterisk (anyone get that close to fixed?), and some problems with returning values because of grep, awk, sed, and/or wc not returning valid numeric values, but overall the idea is solid.
Argus
The current version of Argus has builtin support for monitoring SIP and IAX2, as well as support for querying the asterisk manager interface. Argus can be configured to notify someone if a PRI goes down, or if asterisk goes off to la-la-land (etc).Note: This doesn't appear to be supported in argus-3.3
2005/1/25 13:58:31 17314 ERROR: in file '/var/argus/config' on line 41 - Incomplete specification or unknown protocol for Service SIP
2005/1/25 13:58:31 17314 ERROR: in file '/var/argus/config' on line 40 - Incomplete specification or unknown protocol for Service IAX2
2005/1/25 13:58:31 17314 ERROR: in file '/var/argus/config' on line 39 - Incomplete specification or unknown protocol for Service Asterisk
2005/1/25 13:58:31 17314 ERROR: in file '/var/argus/config' on line 40 - Incomplete specification or unknown protocol for Service IAX2
2005/1/25 13:58:31 17314 ERROR: in file '/var/argus/config' on line 39 - Incomplete specification or unknown protocol for Service Asterisk
Cacti
Cacti is a general purpose monitoring application, one user has developed an Asterisk plugin. More information on this plugin can be found here.Update 2007
We at ITConnection.ru are happy to post a Cacti script for Asterisk IP-PBX statistics. It's based on Python, works over Asterisk manager API connection and provides cute and comprehensive graphs. You can download Cacti's plug-in from this page: http://forums.cacti.net/viewtopic.php?p=111317
Hobbit
Hobbit is a Big Brother compatible network monitor.MRTG
Munin
Nagios
Notes documenting a simple Asterisk monitoring implementation utilizing Nagios:- Nagios check_sip plugin
- Official Nagios Plugins http://nagios.org/download/ have a plugin called contrib/check_asterisk.pl This does remote IAX2 checking as well as remote checking through the manager interface.
- Plugin for monitoring Asterisk with Nagios, download the perl script and change the username, password, and server IP for the manager interface. http://megaglobal.net/docs/asterisk/html/asteriskmonitor.html
- Yet another plugin to allow monitoring of concurrent calls for different protocols, or testing for IAX responsiveness can be found Here
- A Nagios plugin which check the status of zaptel spans: Asterisk Zaptel Nagios plugin
- Dev Nagios IAX2 plugin. Checks from remote, not the Manager Interface. Uses IAX2 POKE. Other options are there just commented out. This works but could be improved. http://odunix.org/pbx/
- A nagios plugin wich checks weather zaptel spans are active or Deactivated http://edd.za.net/checkspans.php
- A nagios plugin compatible with NRPE (the script run on the Asterisk server). Display status of SIP account, Zaptel span... Official page en french - Sourceforge
PacketIsland PacketSmart
Packet Island's PacketSmart http://www.packetisland.com provides an asterisk software agent, a VoIP sniffer micro-appliance and a web hosted management service that allows for detailed SIP, RTP, and call route performance monitoring for Asterisk VoIP calls. MSPs can do remote monitoring and analysis of asterisk client sites using this product.SNMP
The Asterisk team has added SNMP support to Asterisk 1.4.Other ideas
The manager interface sends events when a channel/span goes into alarm.A simple app collecting this data should be easy to write
1. Installing a basic SNMP agent on each Linux box and using a central SNMP manager to monitor each node. This would give notice when a remote node became isolated from the monitoring network.
2. Rolling in Asterisk alarm logs into a syslog server or even as SNMP traps.
3. Writing a small app that simply interrogates those interfaces that are important to the operation (iax2/udp, sip/udp, etc, send a crafted pkt and interpret the returned result. Port not open is obvious, no response is obvious, incorrect response is not so obvious)
4. Test call to an outside number once per five minutes, hourly, or whatever trips your trigger (outside number only needs to respond with something that is predictable, doesn't have to be a person or company)
5. Monitoring logs looking for keywords (may take some time to identify the appropriate keywords) (e.g. Swatch or Logwatch)
It would seem logical for someone to write a res_snmp.c for asterisk that would expose a lot of asterisk's internal data. This would seem a logical step toward writing fully functional monitoring applications as well. The module would allow clients to add themselves to the list and receive traps, as well as check for the current status of various variables.
_Idea is ours and now need to write code....... please include code here ....... Nahid Hossain_
James Thompson: I'd suggest using something like Sipsak to actually do a SIP registration to verify the service is up.
You could use the Whats Up Gold send/expect to connect to Linux box, run sipsak, and then check the results.
<brainstorming>
Okay, this may be over the top, but here goes. Write an asterisk application that sends (and receives) status information to another box over the PSTN. My idea is not only to use this as a way to verify that * is running, but asa way to RELIABLY tell that a remote * box is actively accepting incoming calls. It wouldn't have to be anything complicated, just a heartbeat and some basic details to let the caller know that "yes, I'm alive and acceptingcalls over this line". Simplified protocol:
1) Monitoring box calls up and says (in DTMF):
#<my CallerID>
#<extension I am trying to reach>
#<I'm a machine, so reply in DTMF instead of voice>
#<the secret code is>#
2) The remote box says:
#<your CallerID>#<Your DNIS>
#<yes I will accept a call to thatnumber>#
3) Monitoring box acknowledges and disconnects
4) Remote box disconnects
5) Monitoring box decides whether it likes the answers it received and performs actions accordingly.
</brainstorming>
Hint: Do a "show application AlarmReceiver" and you are very close to what you brainstormed... I think...
TJH- (nicheware)
I've had some problems with peers losing their registrations, usually due to crappy ISP's with DNS problems.
This script tests if peers are up. (registered) I call it from a cron job. If a peer loses it's registration I receive an e-mail.
#!/bin/bash
for peername in peer_1 peer_2 peer_n
do
# note, next lines are "command substitution" - use backticks ` not quotations ' as appropriate
otpt=`/usr/sbin/asterisk -rx 'iax2 show peers' |grep $peername |grep Unspecified`
if [ -n "$otpt" ]; then
echo $otpt | mail -s "Asterisk peer: \"`echo -n $peername `\" has a problem." "you@yourdomain.com"
fi
done
down. Sometimes if their asterisk is just hosed they will still have the same IP- so I can ssh in and fix it. Also it would be
nice to send either an electronic page or even a call file to alert me of problems - although I don't want the phone ringing
at 4AM.
-TJH
HAL
grep has problems with not printable chars, so you should use "strings -a" in your script.
otpt=`/usr/sbin/asterisk -rx 'iax2 show peers' |strings -a| grep $peername |grep Unspecified`
HAL
Another note from a user on a large scale (S McGowan, 2005)
We're using Zabbix to run local UserParameter commands to monitor Asterisk. We use asterisk -rx commands to output what we want and then return the result to the Zabbix monitoring server. There's been small problems with too many remote connections at once crashing asterisk (anyone get that close to fixed?), and some problems with returning values because of grep, awk, sed, and/or wc not returning valid numeric values, but overall the idea is solid.
See also:
- Astricon Europe presentation about asterisk stability and security
- Asterisk administration
- ast-ax-snmpd an old and now obsolete SNMP module for Asterisk
- Network Monitoring
- Asterisk | FAQ | Tips & Tricks

Comments
333check_asterisk plugin doesnot return any output
does anyone got it right before.... :( please help....
is there any way to monitor the concurrent calls in real time
Thanks
Shiju V. Joseph
333Nagios with SMS Server Tools
333Asterisk res_snmp on suse 9.1
<p>Here are my notes on building it on Suse 9.1</p>
<p>The default suse 9.1 install of netsnmp does not 'play-nice'
with the asterisk res_snmp build.
<br/>
Even updating the net-snmp packages (core and devel)
does not work.</p>
<ol>Here's what I did to make it work:
<li>uninstalled net-snmp and net-snmp-devel rpms</li>
<li>downloaded net-snmp-5.3.0.1.tar.gz and unpacked it.</li>
<li>fix some SUSE weirdness in the .so names</li>
<li>built net-snmp with </li>
(note that installing to the default location /usr/local
does not work. The asterisk build assumes the headers
are in /usr/include)
<li> download asterisk svn-trunk and build asterisk</li>
<li> edit the snmpd.conf to say (minimally)</li>
<li>edit /etc/asterisk/res_snmp.conf to say:</li>
<li>now start snmpd </li>
<li>start asterisk </li>
<li>test res_snmp</li>
you should get something that looks like:
</li>
that's the asterisk version you have just built.
</ol>
<p>
I have not run res_snmp for more than a few minutes, so I can't tell how
useful/stable it will be.
</p>
<p>
I plan to write some simple monitoring tools that use snmp when I get the time.....
</p>
333love to see something inside logwatch
We use logwatch a lot here, think it is pretty slick, & I know that it doesnt do realtime monitoring, but great for collecting statistics, and great for having someone come in each morning, glance through a few emails & know what is going on.
If any one gets any asterisk stuff in to logwatch, do please let me know, would be in your debt. I'm not the programmer I wish I was, but I will take a crack at it myself, and post my results.
Peace. Love. Linux.
333More SNMP ideas
There is an SNMP extension to Asterisk, but it's still in what I'd call the "Beta" stages. Please try it out, create patches, and report back to the author.
http://www.faino.it/en/ast-ax-snmpd.html