Laptop Data Backup Service

This is a service for DAMTP people to securely backup data on laptops from any Internet connection (as long as ssh is allowed). You can also use this service for desktops on the laptop network.

To use this service, you will need

  • A connection to the internet.
  • A live DAMTP UNIX account.
  • Administrative permission to the computer (Windows only: needed to install WinSynCMS).

You are working on your laptop in another country while at a conference. You have a wifi connection to the Internet. Would you like your data to be safely and securely backed up to a storage server at DAMTP?
When working on the CMS laptop network would you like regular, scheduled, backups of your data to be taken?
If "yes", to either of the above, then maybe this laptop data backup service is for you.

We welcome feedback about this new service, to help@damtp

Note: if the same person uses this service to backup 2 laptops both with the same computer name those backups will go on top of each other, so we suggest you do not do that.

The backups are "snapshots" so the first backup takes a long time, then subsequent backups are of files that have changed since the last backup.

Windows XP/Vista - WinSynCMS

(Latest = 25/5/2010 version=2.360 )

We provide software (WinSynCMS) for backup of data on Windows XP and Vista laptops over a secure network protocol (SSH) to a DAMTP storage server.

The first backup of your data is a full one. Subsequent backups are incremental for efficiency of time and use of backup storage space. You can select the folders you want to backup and how often (automatic schedule). A method for retrieving files from your backups is also provided.

WinSynCMS uses rsync and ssh from the Cygwin project, and winscp software.

The built-in Help gives more details explaining how it all works, how to set up automatic schedules, the registry entries it uses, file locations important to how it functions, log files etc. A copy of the Windows help file is available for those who wish to read it before downloading the complete program.

Download the program (WinSynCMS is a small download = 8MB)

A portable version is also available. The portable version of WinSynCMS allows you to run the program off a USB device and create a number of profiles that each contain a different set of folders that you'd like to back up onto the same remote server as the 'normal' installed stand alone version of WinSynCMS does. A profile can be created for different computers, for example, one could be for your home pc, another for your laptop, another for your children's computer. Each profile requires you to make an SSH key pair, the profiles don't share one key. Because the USB device contains private SSH keys, all of the profiles and the WinSynCMS program are stored within a secure volume that is created and opened using a third party freeware application called Safehouse Explorer. Download the zip file, (28MB). This contains three files, one of which is a help html file that gives detailed instructions on how to use and run WinSynCMS off a USB device.

NOTE: you will need a live DAMTP UNIX account for WinSynCMS to work.

WinsynCMS Change log

12th February 2009 - First pilot release

23rd February 2009 - Vista version produced and released for testing

17th March 2009 - Added the date and time of the last 'Last Known Good' backup.

27th March 2009 - Fixed a bug that stopped the program from loading if the user hadn't already made a backup. The date and time of the last 'Last Known Good' wasn't being handled correctly if no LKG existed.

21st July 2009 - Changed the way XP calculates the folder size, speeds up the calculation for large folders.

21st July 2009 - Created the portable version of WinSynCMS allowing it to run off a USB device.

17th September 2009 - Improved the 'next schedule' date and time reading. Unreliable output frm schtasks command. Stops 'missed tasks' warning appearing when not the case.

18th November 2009 - Improved the speed of error checking of the rsync log file once backups have completed. If the log file is very large, when a manual backup takes place, users are shown a message box for each stage of the error checking, the text on the 'Manual backup of folder(s) now' button also changes to show what stage of checking it is on. This occurs if the checking takes more than 30 seconds between stages, there are 5 stage of checking in total.

For manual backups, users are given the option to choose the logging level.

For manual backups, users are given the option to turn on and off the 'show log file as a backup takes place'. If displayed, backups take longer.

Users receive a warning if a scheduled backup produces rsync errors. Users can choose to turn this warning off in the system tray icon menu.

6th January 2010 - Added the function for users to export their settings and public/private ssh keys to a password protect zip file. Import function also added.

13th May 2010 - Fixed a bug that stopped backups completing if only a single directory was in the list of dirs to backup. Also added a system tray balloon when the backup has finished.

Please report problems and suggestions to help@damtp, thank you.

Linux and Max OSX - Command Line Method with a script

Using ssh, rsync and cron (for help: man 5 crontab).

Download the script laptop_data_backup.sh [ Last updated 2010-Sep-16 ] onto the computer where the data you want to backup resides, e.g., your laptop (probably). You may want to put it in your home directory or create a dedicated directory for it. In the following we refer to this directory as the ${SCRIPT_DIR} and we use ${CRSID} to refer to your DAMTP UNIX system login (Common Registration Service identifier). So, for example, if your CRSID is js234 and you decide to put the script in a subdirectory of your home directory called tardis, then executing these two lines in a terminal:

CRSID=js234
SCRIPT_DIR=~/tardis

will allow you to copy and paste the script fragments (which follow below) into the same terminal in order to execute them.

Also download and save in the ${SCRIPT_DIR} the following files:

Instructions for use

# Make this script executable (laptop_data_backup.sh) and then run it:

cd ${SCRIPT_DIR}
chmod u+x laptop_data_backup.sh
./laptop_data_backup.sh

# The script will take you through the following steps:
# 1. editing the configuration file: tardis.conf
# 2. creating the file tardis_backups.txt with a list of directories in it to backup
# 3. creating ssh keys and copying them to the laptop backup server
# 4. editing the excludes list if necessary tardis_excludes.txt

# Mac OS X, at least up until v10.7 (Lion), comes with an old and incompatible version of rsync - see tardis.conf
# for more details.  Enter:

rsync --version

# at a command line prompt to check what version you have.  If you need
# to update it, untar this archive rsync-3.0.6-mac-intel.tgz in the root directory
# of your laptop, i.e.:

cd / ; tar xvzf rsync-3.0.6-mac-intel.tgz

# and then set the PATH variable as described in tardis.conf.

# To make this backup run regularly using cron at 18:04 (please choose your own time) each day,
# edit your crontab (with whatever editor ${EDITOR} is set to) by running:

crontab -e

# and add a line like this (but WITHOUT the leading hash character, '#'):
#
# 4 18 * * * ${SCRIPT_DIR}/laptop_data_backup.sh
#
# DO replace ${SCRIPT_DIR} with the actual path to the script in the
# crontab entry above, please.
#
# Please note that using cron means your computer has to be switched on to make
# the backup at the specified time, and connected to the Internet.  Note as
# well that at the specified time the backup will run irrespective of where
# you are. This can cause problems if you have to pay for your data transfer
# per unit transferred (e.g., when using a broadband dongle) and you have
# accumulated a lot of new data (e.g., you've downloaded a film) since the
# last backup was made. To avoid this possibility, and assuming that you are
# happy to have the backup run only when you are connected through the
# University network, use this cron job instead:
#
# 4 18 * * * expr $(hostname) : '.*\.cam\.ac\.uk$' >/dev/null && ${SCRIPT_DIR}/laptop_data_backup.sh

Options for Retrieving Data

SFTP (secure file transfer over SSH)

sftp ${CRSID}@tardis.damtp.cam.ac.uk

nautilus - the Gnome graphical file browser typically on Linux

nautilus --no-desktop ssh://${CRSID}@tardis.damtp.cam.ac.uk/local/tardis/laptops/${CRSID}

Fugu - a MacOSX graphical SFTP client, see the Fugu home site for download and instructions. Also the well known FTP client Fetch will do SFTP protocol.

sshfs (MacOSX and Linux can do this, though you may have to install the software)

# if you have sshfs (fuse package) installed
# to make the mount point for the tardis connection
mkdir /tmp/tardis
# mount the connection
sshfs ${CRSID}@tardis.damtp.cam.ac.uk:/local/tardis/laptops/${CRSID} /tmp/tardis
ls /tmp/tardis # check files are there okay and retrieve if desired
# unmount the directory
cd
fusermount -u /tmp/tardis

Please do NOT mount the fs in a sub-directory of the tree that is being backed up or the backup may descend into the backup and attempt to back that up to the server too. You don't have to be Kurt Gödel to realise that this will probably end in tears!

Release of the software

The software that has been created here to provide this laptop data backup service is now released under the GPL.