.. index:: backups
.. index:: cluster; backups
.. _backups:

Backups
=======

Please contact `Andy Thomas <andy.thomas@imperial.ac.uk>`_ if you need access to old files.  Obtaining files can be a time-consuming process; please only use this as a final resort!  A daily snapshot of /home is taken overnight and stored on the workstations: this should be the first port-of-call for obtaining recent files (see below).

Principal backup
----------------

Files in /home, /data and settings from the NFS server (currently ramsay) are backed up every night using `rsnapshot <http://www.rsnapshot.org>`_ to a NAS box.  For security the NAS box is distinct from the rest of the CMTH computing infrastructure.  We aim to keep 31 days of daily snapshots, weekly snapshots for 8 subsequent weeks and 3 monthly snap[shots thereafter (i.e. going back a total of 6 months).  This might not be possible if large files are changed regularly.  Please do not store large temporary files in filesystems which are backed up; /workspace is a far more suitable place for such files.

Some files are deliberately excluded from being backed up; such files are either typically temporary files, large working files produced by running calculations or (e.g. Dropbox) are automatically synched to the cloud and contain personal files which are not suitable to be backed up on our systems.  The full list is (in rsync syntax, see the rsync man page)::

    - **/Dropbox
    - **/.comsol
    - /data/admin
    - **junk
    - **Junk
    - **/.nfs*
    - **tmp
    - **.tmp
    - **Cache
    - **cache
    - **cache4
    - **Trash
    - **.Trash
    - **Trash.*
    - **Dumpster
    - **spool
    - **/.dt
    - **/core
    - **/core.*
    - **/.netscape/xover-cache/*
    - **/.mozilla/*/News/*/*.msf
    - **/.mozilla/*/Possible\ Spam
    - **/.jpi_cache
    - **/history.dat
    - **/.thumbnails
    - **/.thunderbird
    - **/.evolution
    - **/gnome-terminal-*
    - **/.sawfish/sessions
    - **/.xvpics
    - **/.gimp*
    - **/.gxine
    - **.dvi
    - **.aux
    - **.log
    - **~
    - **/gvfs-metadata

Network backup
--------------

To augment the other backups, we also have a network backup system written by `Matthew Foulkes <http://www.cmth.ph.imperial.ac.uk/people/m.foulkes/>`_.  

Every night, /home is copied into the /backup directories of the workstations in a distributed fashion.  Note that no history is kept---the network backup contains a single snapshot of the files in /home.

To locate the backup of your home directory, run::

    $ yesterday
    Yesterday's squirrel backup found in: ph-jhamm:/backup/rayleigh1/home/jspencer

In this example, the yesterday returns 'ph-jhamm:/backup/rayleigh1/home/jspencer', which means that my home directory from last night's backup is on the workstation ph-jhamm in the directory /backup/rayleigh1/home/jspencer.  To restore your files, log into the given workstation (via ssh or directly) and copy the desired files from /backup/rayleigh1/home/username to your home directory.  The location of your backup directory may change from time to time.

rsnapshot
---------

Rsnapshot is also installed on all workstations so that users can use to to manage backups of their workspace should they desire. **The workspace is not backed up automatically on any machine**. If you are keeping important files there, you will need to manage a backup yourself. Rsnapshot allows you to
automatically save daily/weekly/monthly snapshots to an external server or drive.

Note if you are using an external drive, it is best to format it in a linux format such as ext4,
using for example the utility :code:`gnome-disks`. Rsnapshot makes extensive use of hard links (where more than
one file points to the same data on disk) so that each additional snapshot uses only a minimal amount of additional storage space, so it is best to use
a disk format which allows this.

If you attach an external drive, the script :code:`/common/sbin/setup_local_rsnapshot.sh` can
setup an automated backup of your workspace to this drive for you.