TUTORIAL Image copy Backup (and Restore) of RackNerd VPS HDD

[Halea]

1) Login into your RackNerd VPS account at https://nerdvm.racknerd.com/login.php
2) Pick your VPS server from the list by clicking on its hostname or the corresponding "Manage" button.
3) When the dashboard for the VPS shows up, click on the "VNC" button.
4) On the VNC Viewer page click on "HTML5 VNC Client SSL". (Make sure that you allow web browser window pop-up for nerdvm.racknerd.com)
Alternatively, you can pick the IP address, access port number and temporary password from the screen and use your favorite VNC client to access your machine. Note that this is host level access with a different IP address and user credentials than what you setup inside your VPS. It works as if you are at the physical console of your virtual machine even if its ssh access is damaged or not available.
5) Your web browser will open an HTML5 window with an SSH connection to your VPS' main console.
6) Login with your VPS machine's root credentials.
7) Once logged in, run the command "shutdown -Ph now" to properly shut off your VPS. When done, the VNC window will say that the server is disconnected.
8) Go back to the web browser's VNC Viewer page and click on "Back", which will take you to the VPS dashboard.
9) Next, click on the "Rescue" button, which will take you to a new page, allowing you to select the kernel to be used for the rescue mode.
10) Just use the default kernel shown in the selector as it's the main kernel installed by your VPS at configuration time. If you go into rescue mode for system troubleshooting you can change it for something different, but here we are dropping into rescue just to do a backup. Click on "Enable Rescue Mode". Then confirm in the pop-up notification window that you want to proceed.
11) At this stage you should see SSH connection credentials for root with a hard to guess password. Note that the IP address is your VPS' own static IP.
12) In order to backup your VPS' HDD in its entirety (as a byte per byte image copy), open a terminal on your internet connected local machine and run the following command (locally, not on your VPS instance):
ssh root@<YOUR_VPN_IP> dd if=/dev/vda bs=1M oflag=sync | sudo dd of=<FULL_PATH_&_FILE_NAME_FOR_BACKUP> bs=1M oflag=sync status=progress
Note: In order to restore a backup, run the steps #1 through #11, then use the following command on your local computer (not on your VPS instance):
sudo dd if=<FULL_PATH_&_FILE_NAME_TO_RESTORE> bs=1M oflag=sync status=progress | ssh root@<YOUR_VPN_IP> dd of=/dev/vda bs=1M oflag=sync
13) When the backup or restore operation is complete, exit the rescue mode and reboot your VPS into normal operational mode.

PS:
The steps #1 through #7 are the long way to shut down properly your VPS, but it's a great way to become fully acquainted with the RackNerd system dashboard. (At step #3 you could very well click on "Shutdown" instead of invoking VNC and get the host to power off your VPS through an ACPI call. Or, you could simply ssh into your VPS your favorite way and shut it off using your favorite command.)
The steps #8 through #11 are to start the rescue mode on your VPS. This mode basically powers up your VPS with a generic minimal Linux OS, so that you can do troubleshooting on your machine's HDD while the system installed on the said HDD is not active. This is what you need in order to do your backup safely, as you wouldn't want data to be changed while the transfer out is in progress.
The step #12 makes the actual backup or restore operation to take place. Those are commands run on your local machine, in a standard terminal where ssh and dd are used.
The backup command starts a remote dd session through ssh and diverts the output to the local machine where another dd instance forwards the incoming data stream into a local file.
The restore command on the other hand, starts a local dd instance whose output gets forwarded to the remote machine's dd input via ssh and the resulting stream gets written to the character/bloc device /dev/vda.
With both commands, status=progress allows for real time visualization of the copy progress. oflag=sync is to ensure that the last (possibly fractional) data bloc is flushed onto the device without delay.

 

[wardmundy]

This development coupled with RackNerd's $1 a month VPS makes this an unbeatable combination.

A Fresh Look at Low-Cost VoIP Cloud Providers – Nerd Vittles

nerdvittles.com

 

[Halea]

I just wanted to update this post with another alternative which, although a bit unorthodox, works equally well and has an advantage over the method described on the original post.

See, I usually install Xfce4 on all my servers, whether they are on bare metal or VMs. But I set it up in such a way that it only runs on demand to not waste precious CPU and RAM resources. (I typically use gparted, grub-customizer, boot-repair, thunar file manager, and occasionally fire up firefox to access online knowledge bases as I (re)configure the system)

The alternative approach here for backing up IncrediblePBX is based on Timeshift, which is a neat utility capable of doing live snapshots of the system at requested intervals. The snapshots are stored in a folder that, once completed become totally static and can be copied out without worrying about the live system changes.

So, the advantage over the original method (of shutting down the operational system, firing up the rescue mode and copying out the system partition with dd) is that IncrediblePBX keeps running at all times, there is no down-time!

To test this method, I started by running Xfce4. Surprisingly the resource utilization impact on the overall system was quite negligible. (My Xfce4 installs are lean to begin with). I configured Timeshift to make one daily snapshot on my cheap RackNerd VPS, and I wrote a basic one liner (pretty similar to the one in my original post) to copy out the snapshot folder over ssh to my remote server.

Earlier, I restored the snapshot on a VM in my office and it worked flawlessly. Then I messed up the test PBX on RackNerd just enough that the SIP trunks all failed. Timeshift was able to restore the local snapshot in less then 3 mn and the PBX was up and running again. All these without ever bringing down the system!

So, with cheap VPSes the absence of integrated backup solution doesn't need to be a major downside. Utilities like Timeshift, almost all based on rsync (and could be used directly without Xfce4+Timeshift if one is willing to script it out and run a crontab) work well and provide the desired result.

Also worth mentioning; Timeshift can be used without a GUI, directly from command line. Unfortunately I never use it that way, so I am not able to vouch about the convenience it offers.

Update (3/13/2021): I've tested timeshift's CLI capabilities. It works well and is very simple to use.

A "sudo timeshift" from a terminal is sufficient to generate an immediate snapshot which generates progress information on the console including a clear identification of the directory where the static backup is made. So, it's quite easy to fetch it and move it to a remote backup server. The local copy can be kept and used to go back in time, just like with any snapshot restoration on any system.

The console output could be redirected to a file so that the snapshot relevant information can be preserved. For that a simple "sudo timeshift > timeshift_reportcard.txt" does the job.

As for the remote transfer; if it's initiated on the PBX locally it could be something like:
sudo scp -r -v ~/timeshift/snapshots/2021-03-12_20-50-35 [email protected]:/BackupArea/

If the transfer is run on a remote backup server then the scp line sould look like the following:
sudo scp -r -v [email protected]:/PathToTimeshift/snapshots/2021-03-12_20-50-35 /LocalStoragePath/BackupArea/

Using the CLI approach, it's possible to automate the whole process with a bash script which gets triggered via cron. It would include the step of calling on timeshift to make a local snapshot while the system is live, the step of fetching the reportcard so that the static backup location is identified, then the scp line to get it out to a remote server.

 

[alx359]

Hi Halea,
Great tip, but wish to know a bit more before delving into this rescue feature while my VPS is down.

As it's unfeasible for me to do the VPS backups here at home, do you know if one can apt-get apps like rclone or the self-contained b2-cli to backup to a b2 bucket instead, for example? Also, are any file changes we may make inside such minimal rescue setting (keys, login data, etc.) still private to our user/account and persisted between sessions? Thanks!

 

[Halea]

... do you know if one can apt-get apps like rclone or the self-contained b2-cli to backup to a b2 bucket instead, for example?

I suspect that rclone will also work fine, although I haven't tested it in the specific setup that my original post has referred to. Obviously you'll have to install rclone first on the rescue session. Just bear in mind that the rescue session has very limited hard drive storage, so you have to be frugal in the choice of your tools.
Tools such as rsync, rclone, dd, dcfldd, dc3dd take little space.

Also, are any file changes we may make inside such minimal rescue setting (keys, login data, etc.) still private to our user/account and persisted between sessions? Thanks!

That largely depends on your provider's rescue disk scheme. With RackNerd for instance there is no such capability.
In fact, in their case the rescue disk is nothing more than a very basic live OS on which a RAM based file system is overlaid. The minute you exit it your custom data is lost.
You can probably write a script which automates the install of the extra software that you need for your rescue/backup job, and that could include some configuration material. But again, the limited space becomes an issue quickly.

This is why I like partitioning the main disk and have my own "loader" partition which only serves as a glorified rescue disk. And when I have enough storage I lavishly set it up with a full-blast desktop like Xfce.
In the old days of Linux Mint 17.x (or earlier) it was possible to squeeze a full blown Xfce desktop with tons of utilities in as little as 8GB. With today's bloated OS distros it's no longer possible (with GUI).

 

[dicko]

One solution is to follow

Clonezilla Live on hard drive

to install as a grub boot option. That way you can remotely clone your system (downtime needed) then later restore it either by rebooting into clonezilla on the broken system or booting the image into a 'clonezilla live' iso on another bit of hardware if completely effed up.

Another solution is to look into

Mondo Rescue - GPL disaster recovery solution

Mondo Rescue

www.mondorescue.org

That needs no downtime to clone , a little downtime to restore.

Of course in 2022 most cloud systems have this all built in for a dollar/month or so

 

[alx359]

@Halea: thanks for elaborating on your experience with the rescue feature. The racknerd guys also replied with a livecd analogy to convey similar limitations.

@dicko: will look into these, too. At glance they seem quite more involved, and beyond my current zone of comfort with sysadmin. Especially if I mess something up, or bumped into an odd case, etc. That's why I'm leaning toward the simplicity of rclone or b2-cli.

Thanks again both for responding.

 

[jeffshead]

I realize this is an old thread but it's the only Internet search result which contains detailed instructions on performing a full backup of a Racknerd VPS, remotely.

@Halea - Your instruction are awesome! However, I would like to perform a backup/restore where the backup image is smaller because it's not a raw clone of the entire disk. I have some Racknerd VPS's that have larger HDD's with lots of empty space. I'm assuming the Clonezilla Live on Hard Drive method would be one of the best alternative methods but I just can't get it to work.

Has anyone made or come across a tutorial for installing and using the Clonezilla Live on Hard Drive method or an alternative method for which a 5 year old could understand