I recently had a broadband outage. While I was waiting to be re-connected to the world, I started thinking about outages in general, and disaster recovery in specific. As I posted elsewhere I backup my computer on each and every day I use it, to two external devices. But what I hadn’t done so far was a test restore from the backup to prove that the system worked.
I know you can do an entire snapshot of your Linux box using Clonezilla, which is a free and open source equivalent to Ghost. But the time taken to generate a snapshot and, more to the point, to restore one versus the time taken to do a fresh install of Ubuntu on my PC seemed too great. My theory was I could do a completely new install of Ubuntu, restore my data and do the small amount of customisation I do to my system in less time than it would take to do a restore of an image and then do a data restore on top of that.
Back in the Windows world it was different. The time taken to install all of my applications added greatly to the restore process, and the customisations and preference settings for each one added further delay to the restore time. it was also a manually-intensive and therefore error prone process. Given those circumstance I did use to image my PC using Acronis True Image. But with Ubuntu because most of the applications I use are installed by default anyway there is no further time lost in getting those back on the PC. And as for the customisation or preference settings it should just be a case of copying the appropriate hidden folders from my backup into my home drive. Shouldn’t it?
I concluded that if there was a flaw with my back up process it would be better to find out now rather than when I’ve had a real-world disaster and need to get things back up and running fast. Time to bite the bullet. I re-installed Ubuntu and did a format during the process. Bullet well and truly bitten.
40 minutes later Ubuntu was up and running on my machine again. I’d given it the same name, gave myself the same user name, and used the same user password.
Obviously I had access straightaway to the stuff I have in the cloud, such as in Evernote. I copied my data back to my home folder: Documents, Music, Videos and so on. I fired up Firefox and logged into Firefox Synch, and retrieved all of my bookmarks and shortcut settings. I closed Firefox and copied the .mozilla folder from the backup drive to my home folder. Any file or folder whose name starts with a . is hidden. you need to hit Ctrl-H to toggle their visibility. I fired up Firefox again and it found all of my add-ons and their settings in the .mozilla folder.
I installed and started Thunderbird and then closed it down straight away. I copied the .thunderbird folder from my backup to my new home folder, restarted Thunderbird and all of my email accounts were there, with their settings and all of my email archives. Also the plug-ins I use in Thunderbird were all present and correct.
I was on a roll. Next: the same thing for FileZilla. Copied the backup .filezilla folder to my home folder, started FileZilla and it had all of my FTP accounts in it with their connection settings, passwords etc.
And so it went on. Application after application had their settings and data dragged back and that was that – they were restored and set up in just the way I’d left them. Once my applications were back on the PC I then set up things like RecollIndex in Cron, and sSMTP and I did some of the visual customisations to the desktop that I like, and that was that. Obviously things would have been a bit more complicated if I had multiple users installed on the system, but as far as single user system restores go, that was a doddle. It was the easiest system restore I’ve ever been involved in.
And believe me, I’ve done my share of them.
One of the best things about open source software is the support. Most packages have active and enthusiastic communities and support forums. I had a couple of questions about luckyBackup.
One question was whether I could disable the function within luckyBackup that deletes files from the destination if they are no longer in the source. If you want to keep the source and the destination synchronised this is great, but it wasn’t what I wanted. If I accidentally deleted a file from the source and then made a backup without noticing, the same file would be deleted from the destination.
The second question was to do with a folder I have that contains video files. This was getting backed up each time I ran the backup task, even when no files had changed. Because the video files are large, it took a long time to complete the backup task so i didn’t want it to run unnecessarily.
I attempted to get answers to my questions by joining the luckyBackup Help Forum on SourceForge but for some reason it was playing up that day. I couldn’t get it to accept any of the user names I tried to register with. So, no immediate help coming from that direction.
However, back on the luckyBackup web site the author of the software had been so kind as to leave his email address and he encouraged people to contact him. His name is Loukas Avgeriou. I dropped him an email. Within a few hours he had replied and answered my two questions. Try doing that with commercial software companies!
As it turned out the remedies were embarrassingly simple. If I’d read the manual a little more closely I’d have probably found the answers myself. If I’d done that though I wouldn’t have a reason to typing this post and letting you know just how inordinately cool guys like Loukas Avgeriou are. To develop something like luckyBackup is a major undertaking. I know, I’m a developer by trade. To do it for free and then happily provide support is quite amazing. Kudos to everyone who contributes to any of the many open source projects.
The solution to my issues with luckyBackup were as follows. To stop luckyBackup deleting files from the destination that are no longer in the source, highlight the task in luckyBackup and then click the Modify button. Click the Advanced button and select the Command Options tab. Uncheck the Delete files on destination check box. Yes, it was that simple.
The issue about the files being sent to the destination even when they hadn’t changed was due to the fact that the destination drive in my case was formatted in a Windows format. The Windows file system cannot handle all of the attributes that Linux can. There is also a difference in the way timestamps are set and checked. To cope with this, luckyBackup has an option so that you can let it know that you are targeting a a Windows formatted destination. luckyBackup will then use a slightly different method of determining which files need to be backed up.
To set the option you need to highlight the task and click the Modify button. Click the Advanced button and select the Command Options tab. Make sure the the Destination is FAT/NTFS option is checked.
I backup at the end of each and every day that my computer has been used. You should too.
I prefer backups that move copies of my files onto the backup media in the same tree structures as the sources files were in, and copies them as plain files. This means they are easily accessible whenever and where ever I mount the backup media. If I want to copy some of my files on a different computer I can connect the backup drive to it and copy the files over using the OS. I don’t have to install the backup software on that other computer in order to perform a selective restore. After all, the other computer might be running a different OS, and have no access to the backup software that created the backup.
You can instruct most backup software to create backup ‘volumes’ where the backup is actually a catalog file (or a set of such files). A catalog file is a single file holding many of your backed up files in a compressed format within it. They may or may not be encrypted. There is a need and a purpose for this type of backup as well but for me, for my personal use, a simpler approach is best.
On Windows I used to use the excellent (and free) Cobian Backup. Amongst its many options were the ones that would make it perform the way I wanted my backups to be made, and it never failed me. So for me, it was perfect.
That covered the data. I also used to use the excellent (but not free) Acronis True Image. This was one of the few packages I used that wasn’t freeware or open source. It allowed me to take snapshots of my entire hard disk. In case of catastrophe I could restore the entire platter in one step. I would then restore my most recent data backup and I’d be back to where I had been before the disaster. It hauled my ass out of the fire on more than on occasion.
Making the effort to create the periodic images took time and a degree self-discipline, and restoring from them could take anything up to an hour on my system. Because my PC is pretty fast I can re-install Ubuntu in about 20 minutes, so I’m not planning on making images of my hard disk. If the worst comes to the worst I’ll format it, re-install Ubuntu and simply start again. It’s faster than re-imaging. There are some perfectly good Linux based equivalents of Acronis, such as Clonezilla, that would allow me to image the whole hard disk for disaster recovery, but I don’t think they would save me any time when it came to the crunch.
Of course, once I’d restored Ubuntu I would have to install any software I use that isn’t part of the base install, but even taking that into account I think imaging my machine is a thing of the past.
So all I had to do was find a backup application on Ubuntu that I liked, and that backed up the way I wanted. I’ve settled on the bizarrely named luckyBackup. That’s a lower case l at the start of the name. I must admit to feeling a little unsettled about using a backup program with the word lucky in its title, but it has been great. It is sufficiently flexible to allow me to set it up the way I want to use it, and it runs along at a great rate of knots.
The first backup obviously took some time, because everything had to be backed up. But subsequent backups only copy files that have changed or been created since the last backup, and the backups are over in the blink of an eye. You can install luckyBackup using the Ubuntu Software Center.
The luckyBackup application is is actually a front end to the rsync command.
To see the instructions for using rsync directly, open a terminal window and type:
I’m a command prompt man at heart, but just have a read through that and you’ll see why a simplified GUI front-end is sometimes a good idea.