Incredible Backup & Restore Beta

[wardmundy]

WARNING: THIS FIRST BETA HAS BEEN SUPERSEDED BY THE ONE BELOW. PLEASE GO THERE FOR THE LATEST BUILD WHICH NOW INCLUDES A RESTORE MODULE!

We're inching toward a release of the IncredibleBackup and IncredibleRestore scripts for next week on Nerd Vittles. The backup piece is just about finished if anyone would like to try it. We strongly recommend an off-site location for the backups using a device such as the PogoPlug. Do NOT use /root, /etc, or /var trees as your backup location!!

To use the apps, just download the tarball onto your Incredible PBX server. Then edit the scripts and plug in the directory location for your backups. HINT: Be sure the file location is mounted! Then run the script. DO NOT RESTORE A BACKUP TO YOUR ORIGINAL SERVER YET! DO NOT RESTORE A BACKUP TO YOUR ORIGINAL SERVER YET!
This needs further testing before anyone tries that.

The tarball is attached in a supplemental message below.

Once downloaded:

cd /root
tar zxvf incredible.tar.gz
nano -w incrediblebackup  [insert backup location]

Be advised that this backup procedure takes your server off line briefly. On a new Incredible PBX, the off-line time is just over a minute using an Aspire Revo. The complete backup process takes about 3.5 minutes. Backup size on a fresh install is about .5 gigs. Current time stamp is used as the file name for the backup tarball.

Suggestions are most welcomed!!!!

 

[briankelly63]

Nice Ward! Glad to see the "Lab" keeps producing!

Brian

 

[The Deacon]

Works GREAT! My backup initially took about 4.5 minutes to do the backup and shove the backup out to the pogoplug.

Can't wait for the restore script (I can try it on my backup Revo).

AWESOME AS ALWAYS, Ward!

-Rick

 

[wardmundy]

All new design. More along the lines of how Proxmox handles backups and restores rather than Mondo.

Actually, The Incredible PBX was the first real test of this. That's how those systems are built.

 

[ou812]

What about regular PIAF installs, will this process work or is it only for Incredible PIAF PBX installs.

Gary.

 

[wardmundy]

If you know what apps you've added, it could be tailored for that. Up to now, we haven't kept track of apps folks have added, but that may change. It will certainly restore FreePBX settings and all your stuff in the web tree. The article will go into more detail about this.

 

[ou812]

Thanks for the reply thats great to hear, I like to try and keep apps to a minimum on a production machine as piaf has proven to be a reliable distro. Thank's again to the hole piaf team.

gary.

 

[wardmundy]

What we're trying to get to is what I call the "10-year PBX." We've had them for years in the commercial and government sectors, just never in open source. Once you have a functioning, reliable, secure phone system that meets all of your requirements, why would you want to screw with it every six months. Crazy!

So the idea with this backup system is to have your initial install disk and a backup. Then you should be able to create an all-new system that mimics everything on your old system even when the hardware changes/fails. And we want this to work equally well if you want to migrate from a stand-alone system to a Proxmox virtual machine.

The testers we really need are folks with a production machine and a Proxmox server on which we can create and blow away test images that mimic your production system... unless you happen to have two Aspire Revo units sitting around.

HINT: The very worst place to test a new backup solution is by restoring to your production machine before we get this thoroughly tested. As our pioneers will tell you, it usually takes a few days (and sometimes weeks) to work out all the kinks. But we will get there.

 

[wardmundy]

IncredibleBackup + IncredibleRestore

We're making some progress. Need a few pioneers.

NOTE: Latest FIFTH beta tarball is now included here as well as in last message! Fixes included addition of /tftpboot and /etc/sysconfig/network in backups. Network and email restore components now work... we hope.

Only use the IncredibleRestore beta module if you have a second (separate) machine or second Proxmox image or virtual machine where damage can be minimized if something goes horribly wrong!!! Proxmox OpenVZ does NOT work so don't try it. Proxmox KVMs seem to work fine, e.g. from dedicated hardware to Proxmox KVM is OK.

Network and Email restore pieces do not yet work so be sure network connectivity and email are already working on your second machine.

1. Make a backup of your original machine using IncredibleBackup in this archive... not the first or second beta!!!
2. Build a second (functioning) PIAF machine and run update-scripts and update-fixes.
3. Copy the incrediblerestore script to the second machine.
4. Either copy the backed up tarball to the new machine in a separate, dedicated directory or use PogoPlug for backup file storage. Do NOT use /root and /var/www trees!
5. Plug in the location of the backup tarball directory in incrediblerestore script.
6. If your original system uses a different version of FreePBX than 2.5 (which comes with PIAF base install), then be sure to choose to restore FreePBX.
7. If your main system is an Incredible PBX or Orgasmatron build, be sure to choose to restore Incredible Apps.
8. If your main system is still on line, be sure to disable outbound SIP and IAX so that your registrations don't get confused by having 2 machines registering to same providers. The script will do this for you and explain how to reverse it down the road.

Please report back your progress!!! Good luck and thanks.

To download, log into your server as root and...

cd /root
wget http://incrediblepbx.com/incredible.tar.gz
tar zxvf incredible.tar.gz

 

[ou812]

Just finished loading to a test box and it seems OK.
Original system was a Intel MB with attom330 and 2 gig ram and a 4 port analog digium card. test box is same MB with 1 gig ram and no digium card.

ORIGINAL BOX
* Running Asterisk Version : Asterisk 1.4.21.2
* Asterisk Source Version : 1.4.21.2
* Zaptel Source Version : 1.4.12.1
* Libpri Source Version : 1.4.9
* Addons Source Version : 1.4.7
********************************************************************
pbx.local on 192.168.123.150 - eth0
CentOS release 5.3 (Final) :32 Bit Kernel: 2.6.18-128.el5



root@pbx:~ $ ./incrediblebackup
IncredibleBackup is free software PROVIDED AS IS subject to GPL2 license.
NO WARRANTIES EXPRESS OR IMPLIED ARE PROVIDED WITH THIS SOFTWARE
INCLUDING THE IMPLIED WARRANTIES OF FITNESS FOR USE AND MERCHANTABILITY.
YOU AGREE TO ASSUME ALL RISK AND TO DETERMINE SUITABILITY TO MEET YOUR NEEDS.
Before continuing, read the license:
http://www.gnu.org/licenses/old-licenses/gpl-2.0.html
To proceed at your own risk and agree to the license, press Enter. Otherwise, Ctrl-C.
Incredible Backup starting...
Backing up current FreePBX distro version to /root...
--14:46:20-- http://mirror.freepbx.org/freepbx-2.7.0.tar.gz
Resolving mirror.freepbx.org... 69.41.162.148
Connecting to mirror.freepbx.org|69.41.162.148|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 6496526 (6.2M) [application/x-gzip]
Server file no newer than local file `freepbx-2.7.0.tar.gz' -- not retrieving.

Stopping services...



STOPPING ASTERISK
Asterisk Stopped

STOPPING FOP SERVER
Shutting down Flash Operator Panel 2: [ OK ]
Stopping MySQL: [ OK ]
Stopping httpd: [ OK ]
Creating tarball...
tar: Removing leading `/' from member names
tar: Removing leading `/' from hard link targets
tar: Removing leading `/' from member names
tar: Removing leading `/' from member names
tar: Removing leading `/' from member names
tar: Removing leading `/' from member names
tar: Removing leading `/' from member names
Starting services...
Stopping httpd: [FAILED]
Starting httpd: [ OK ]
Stopping MySQL: [FAILED]
Starting MySQL: [ OK ]


SETTING FILE PERMISSIONS
Permissions OK

STARTING ASTERISK
Asterisk Started

STARTING FOP2 SERVER
Starting Flash Operator Panel 2: [ OK ]
Compressing tarball...
Done.
Moving 1273171580.tgz backup to /root/gary directory...

IncredibleBackup is done.


TEST BOX
* Running Asterisk Version : Asterisk 1.4.21.2
* Asterisk Source Version : 1.4.21.2
* Zaptel Source Version : 1.4.12.1
* Libpri Source Version : 1.4.10.2
* Addons Source Version : 1.4.7
********************************************************************
pbx.local on 192.168.123.13 - eth0
CentOS release 5.4 (Final) :32 Bit Kernel: 2.6.18-164.6.1.el5


The tftpboot file is empty on test box. I think this should be included.

gary

 

[wardmundy]

I'll add /tftpboot.

Might be worth looking through the list of directories that get backed up in case I missed anything else.

Right now none of the card-specific stuff gets backed up either unless it's in /root. Just need some hints if we want that added.

In your case, you obviously would NOT want the card-specific stuff restore (for example to a virtual machine) so we could make the restore optional. We plan to do something similar for network and email settings.

Thanks.

 

[The Deacon]

I did a backup on one Acer Revo (production) and restored that backup to another Acer Revo (stage).

The initial restore went very well (all the preliminary tests appears to work flawlessly).

I too noticed that the /tftpboot directory was empty, but that's already being addressed.

One file we might want to add (and this might be a moot point once the networking / e-mail piece is completed) is /etc/sysconfig/network file; since the hostname is set there.

The real test comes tomorrow morning when I do the backup / restore and then promote the stage box to production.

-Rick

 

[MyKroFt]

Gotta look when i get home, but is the MOH dir being grabed? For those of us with custom MOH?

 

[wardmundy]

Yep. Beta #4 bleow gets /tftpboot. The /var/lib/asterisk tree already is captured. That would include music on hold directories.

I got some of the /etc/sysconfig stuff but missed network so it's been added.

For network restore (which isn't functional yet), this #4 beta now grabs:

/etc/hosts /etc/resolv.conf /etc/sysconfig/network-scripts/ifcfg* /etc/sysconfig/iptables /etc/sysconfig/network

When you do a network restore, all of the network stuff will be found in a restored tarball living in / folder, e.g. /1273154678.1.tar
In the fourth release (below), the mail restore is split apart from the network restore into a new /1273154678.2.tar.
Both are deleted after the restore finishes so that we don't get a mess when you make a net backup of your old restore.

NOTE: New tarball #4 below is identical to the one now posted on the first page of the thread. But the instructions for use are there.

 

[wardmundy]

Almost Final

OK. Here's #5 with all the pieces working including separate email and network restore components. You'll have to make a new backup with the included incrediblebackup script to try this, however, because the network and mail pieces have been separated into separate tarballs internally.

Log into your server as root and issue the following commands:

cd /root
wget http://incrediblepbx.com/incredible.tar.gz
tar zxvf incredible.tar.gz

 

[The Deacon]

This morning I did a bare metal install to my stage box. Installed iPBX up to (and including) the update-scripts and update-fixes point.

Ran the backup (with the most recent version of the scripts) on the production box. Copied the backup file and the new scripts over to the stage box.

Unplugged the production box, did the restore on the stage box and rebooted. The stage box is now the production box (production host name and IP address). I changed the SIPDefault.conf file to use a different bmp file on the phones so I can tell if it is connecting to the new or old production box). I reboot a couple of the Cisco phones and they get the new .bmp file so I know they are talking to the new box.

When I go to make a call, I get the "all circuits are busy" message. I check the admin page and the handsets have all registered, but none of the trunks appear to be registered. Rebooted the box, still no trunks are registered.

-Rick

 

[wardmundy]

In the restore process, sounds like you chose the option to block SIP/IAX outbound.

Check /etc/sysconfig/iptables and see if the end of the file looks like this:

# Disable SIP outbound services
-A OUTPUT -p udp -m udp --dport 5060:5082 -j DROP
-A OUTPUT -p udp -m udp --sport 5060:5082 -j DROP
# Disable IAX outbound services
-A OUTPUT -p udp -m udp --dport 4569 -j DROP
-A OUTPUT -p udp -m udp --sport 4569 -j DROP
COMMIT

If so, there should be a file in /etc/sysconfig called iptables.sip. Just do the following to reactivate SIP and IAX outbound services:

cd /etc/sysconfig
cp iptables.sip iptables
service iptables restart

There also was a nasty little gotcha in version 4. I forgot that /etc/sysconfig/network-scripts/ifcfg-eth0 included a line like this:

HWADDR=6E:76:4A:11:C2:09

If you find that line in yours, just comment it out with a # at the beginning of the line and then service network restart.

 

[The Deacon]

I checked the /etc/sysconfig/iptables files and nothing is disabled.

# Allow connections to our IAX server
-A INPUT -p udp -m udp --dport 5000:5082 -j ACCEPT
# Allow connections to our SIP server
-A INPUT -p udp -m udp --dport 10000:20000 -j ACCEPT
# Allow connections to our SIP-RTP server
-A INPUT -p tcp -m tcp --dport 4445 -j ACCEPT
# Allow connections to our FOP server
-A INPUT -p tcp -m tcp --dport 5038 -j ACCEPT
# Allow connections to xTide web server
-A INPUT -p tcp -m tcp --dport 88 -j ACCEPT
# Allow connections to xTide web server
-A INPUT -p tcp -m tcp --dport 88 -j ACCEPT
# Allow connections to NTP time server
-A INPUT -p udp -m udp --dport 123 -j ACCEPT
# Allow connections to our Alternative Web server
-A INPUT -p udp -m udp --dport 69 -j ACCEPT
# Allow connections to our Webmin server
-A INPUT -p tcp -m tcp --dport 9022 -j ACCEPT
COMMIT

the /etc/sysconfig/iptables.sip file isn't there and the /etc/sysconfig/network-scripts/ifcfg-eth0 looked fine.

 

[wardmundy]

A few silly questions...

What is SIPDefault.conf?

Does the new box have network connectivity outside the LAN?

What does the CLI say when you try to place an outbound call?

Do inbound calls work?

Are you using Incredible PBX with Google Voice and sipgate/IPkall for calls?

 

[The Deacon]

A few silly questions...

What is SIPDefault.conf?

Does the new box have network connectivity outside the LAN?

What does the CLI say when you try to place an outbound call?

Do inbound calls work?

Are you using Incredible PBX with Google Voice and sipgate/IPkall for calls?

SIPDefault.cnf is the default configuration file for the Cisco 79xx series phones that tell it what version of the firmware to load, the proxy server, which timezone to use, as well as which .bmp image to load to the phones.

I do have network connectivity outside the LAN. I can ping / push / pull files to/from work.

I am using iPBX with GV, SIPgate and Vitelity.

Inbound calls are not working either. Here is what I see when I get an inbound call:

NoOp("SIP/64.2.142.13-b7805be8", "Received incoming SIP connection from unknown peer to 707XXXXXXX") in new stack
    -- Executing [707XXXXXXX@from-sip-external:2] Set("SIP/64.2.142.13-b7805be8", "DID=707XXXXXXX") in new stack
    -- Executing [707XXXXXXX@from-sip-external:3] Goto("SIP/64.2.142.13-b7805be8", "s|1") in new stack
    -- Goto (from-sip-external,s,1)
    -- Executing [s@from-sip-external:1] GotoIf("SIP/64.2.142.13-b7805be8", "0?from-trunk|707XXXXXXX|1") in new stack
    -- Executing [s@from-sip-external:2] Set("SIP/64.2.142.13-b7805be8", "TIMEOUT(absolute)=15") in new stack
    -- Channel will hangup at 2010-05-07 16:59:55 UTC.
    -- Executing [s@from-sip-external:3] Answer("SIP/64.2.142.13-b7805be8", "") in new stack
    -- Executing [s@from-sip-external:4] Wait("SIP/64.2.142.13-b7805be8", "2") in new stack
  == Parsing '/etc/asterisk/manager.conf': Found
  == Parsing '/etc/asterisk/manager_additional.conf': Found
  == Parsing '/etc/asterisk/manager_custom.conf': Found
  == Manager 'admin' logged on from 127.0.0.1
    -- Executing [s@from-sip-external:5] Playback("SIP/64.2.142.13-b7805be8", "ss-noservice") in new stack
    -- <SIP/64.2.142.13-b7805be8> Playing 'ss-noservice' (language 'en')
  == Manager 'admin' logged off from 127.0.0.1
    -- Executing [s@from-sip-external:6] PlayTones("SIP/64.2.142.13-b7805be8", "congestion") in new stack
    -- Executing [s@from-sip-external:7] Congestion("SIP/64.2.142.13-b7805be8", "5") in new stack
  == Parsing '/etc/asterisk/manager.conf': Found
  == Parsing '/etc/asterisk/manager_additional.conf': Found
  == Parsing '/etc/asterisk/manager_custom.conf': Found
  == Manager 'admin' logged on from 127.0.0.1
  == Manager 'admin' logged off from 127.0.0.1
  == Spawn extension (from-sip-external, s, 7) exited non-zero on 'SIP/64.2.142.13-b7805be8'
    -- Executing [h@from-sip-external:1] NoOp("SIP/64.2.142.13-b7805be8", "Hangup") in new stack
    -- Executing [h@from-sip-external:2] Set("SIP/64.2.142.13-b7805be8", "DID=s") in new stack
    -- Executing [h@from-sip-external:3] Goto("SIP/64.2.142.13-b7805be8", "s|1") in new stack
    -- Goto (from-sip-external,s,1)
    -- Executing [s@from-sip-external:1] GotoIf("SIP/64.2.142.13-b7805be8", "0?from-trunk|s|1") in new stack
    -- Executing [s@from-sip-external:2] Set("SIP/64.2.142.13-b7805be8", "TIMEOUT(absolute)=15") in new stack
    -- Channel will hangup at 2010-05-07 17:00:07 UTC.
    -- Executing [s@from-sip-external:3] Answer("SIP/64.2.142.13-b7805be8", "") in new stack
  == Spawn extension (from-sip-external, s, 3) exited non-zero on 'SIP/64.2.142.13-b7805be8'
  == Parsing '/etc/asterisk/manager.conf': Found