rCloner

Taking FULL advantage of ‘free’ software is something I love to push the bounds of. Given that the largest data-set I manage is our file-server, I decided to leverage google drive (with free unlimited storage) to make daily backups. One tool later and I am currently making daily full folder compressed and ecrypted backups of the most used folders on our local NAS. At some point below you are going to start asking yourself some questions like “why does it run FROM the production server”, “why pigz?!” and “what colour does a smurf turn when you strangle it”. Why from production server? I wanted to be able to change the backup server to whatever had the prerequisite software and setup done without having to migrate the actual script. Just change the global variables and you’re away. Why pigz? Cause this is for why! Deep-purple.

Prerequisites:

  • ssh-copy-id from the server with your live data to your backup server
  • install and set up rclone according to the rclone docs found here
  • for mail make sure you can use a SMTP relay server or set up mail with credentials

The steps taken are:

  1. generate session hash
  2. rsync production to backup site
  3. compress directory
  4. encrypt directory with session hash
  5. send to google drive
  6. clean up your mess

All the while using mail for notifications.

Tools:

  • openssl
  • rsync
  • tar
  • pigz
  • gpg
  • rCloner
  • mail

backup.sh:

#!/bin/bash

#set globals
LOCALBACKUPDIR=[SET LOCAL BACKUP DIR HERE]
BACKUPSERVER=[SET SERVER IP/HOSTNAME HERE]
DIR=[SET DIRECTORY ON BACKUPSERVER TO USE FOR RSYNC]
ARCHIVEDIR=[SET DIRECTORY ON BACKUPSERVER TO USE FOR THE GENERATE ARCHIVES]
GOOGLEDRIVEDIR=[SET GOOGLEDRIVE DIRECTORY]
NEWLINE=$'\n'

#generate timestamp
NOW=$(date '+%Y-%m-%d')

#generate password
echo 'creating hash'
HASH=$(openssl rand -base64 20) 

#store $HASH somewhere safe - DO NOT IGNORE THIS PART OR YOU WONT BE ABLE TO DECRYPT YOUR BACKUP ARCHIVE

mail -s "Server backup : $NOW" yourname@yourorg.com <<< "$NOW - Starting Backup of GPISERVER. R-Syncing to $BACKUPSERVER"
RSYNC=$(rsync -avz $LOCALBACKUPDIR/* user@$BACKUPSERVER:/$DIR/)

mail -s "Server backup : $NOW" yourname@yourorg.com <<< "$NOW - Compressing and encrypting server backup - ${NEWLINE}---Logs---${NEWLINE}$RSYNC"
ssh user@$BACKUPSERVER "tar c $DIR/ | pigz -1 -p 30 | gpg --batch --symmetric --passphrase $HASH > '$ARCHIVEDIR/$NOW.tar.gz.pgp'"

mail -s "Server backup : $NOW" yourname@yourorg.com <<< "$NOW - Pushing archives to google drive."
ssh user@$BACKUPSERVER "/usr/bin/rclone copyto /$ARCHIVEDIR/$NOW.fin.tar.gz.pgp gd:/$GOOGLEDRIVEDIR/$NOW/$NOW.tar.gz.pgp"

mail -s "Server backup : $NOW" yourname@yourorg.com <<< "$NOW - Cleaning up."
ssh user@$BACKUPSERVER "rm /$ARCHIVEDIR/$NOW.tar.gz.pgp"

mail -s "Server backup : $NOW" yourname@yourorg.com <<< "$NOW - Backup complete."