backup

Secure Automated Backups of a Linux Web Server with Rrsync and Passwordless Key Based Authentication

Backups Automated and Secure

Backing up data is an essential task, yet it can be cumbersome and requires some work. As most people are lazy and avoid tedious tasks wherever possible, automation is the key, as it allows us dealing with more interesting work instead. In this article, I describe how a Linux Web server can be backed up in a secure way by using restricted SSH access to the rsync tool. I found a great variety of useful blog posts, which I will reuse in this article.

This is what we want to achieve:

  • Secure data transfer via SSH
  • Passwordless authentication via keys
  • Restricted rsync access
  • Backup of all files by using a low privileged user

In this article, I will denote the client which should be backed up WebServer. The WebServer contains all the important data that we want to keep. The BackupServer is responsible for fetching the data in a pull manner from the WebServer.

On the BackupServer

On the BackupServer, we create a key pair without a password which we can use for authenticating with the WebServer. Details about passwordless authentication are given here.

# create a password less key pair
ssh-keygen -t rsa # The keys are named rsync-backup.key.public and rsync-backup.key.private

On the WebServer

We are going to allow a user who authenticated with her private key to rsync sensitive data from our WebServer to the BackupServer, This user should have a low privileged account and still being able to backup data which belongs to other users. This capability comes with a few security threats which need to be mitigated. The standard way to backup data is rsync. The tool can be potentially dangerous, as it allows the user to write data to an arbitrary location if not handled correctly. In order to deal with this issue, a restricted version of rsync exists, which locks the usage of the tool to a declared directory: rrsync.

Obtain Rrsync

You can obtain rrsync from the developer page or extract it from your Ubuntu/Debian distribution as described here. With the following command you can download the file from the Web page and store it as executable.

sudo wget https://ftp.samba.org/pub/unpacked/rsync/support/rrsync -O /usr/bin/rrsync
sudo chmod +x /usr/bin/rrsync

Add a Backup User

First, we create a new user and verify the permissions for the SSH directory.

sudo adduser rsync-backup # Add a new user and select a strong password
su rsync-backup # change into new account
ssh rsync-backup@localhost # ssh to some location e.g.  such that the .ssh directory is created
exit
chmod go-w ~/ # Set permissions
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

Create a Read Only of the Data You Want to Backup

I got this concept from this blog post. As we want to backup also data from other users, our backup user (rsync-backup) needs to have read access to this data. As we do not want to change the permissions for the rsync-backup user directly in the file system, we use bindfs to create read only view of the data we want to backup. We will create a virtual directory containing all the other directories that we want to backup. This directory is called `/mnt/Backups-Rsync-Readonly`` . Instead of copying all the data into that directory, which would be a waste of space, we link all the other directories into the backup folder and then sync this folder to the BackupServer.

One Time Steps:

The following steps create the directory structure for the backup and set the links to the actual data that we want backup. With this method, we neither need root, sudo or any advanced permissions. We simply create a readonly view of the data where the only user with access is rsync-backup.

sudo apt-get install acl bindfs # Install packages
sudo mkdir /mnt/Backups-Rsync-Readonly # Create the base directory
sudo chown -R rsync-backup /mnt/Backups-Rsync-Readonly # Permissions
sudo mkdir /mnt/Backups-Rsync-Readonly/VAR-WWW # Create subdirectory for /var/www data
sudo mkdir /mnt/Backups-Rsync-Readonly/MySQL-Backups # Create subdirectory for MySQL Backups
sudo setfacl -m u:rsync-backup:rx /mnt/Backups-Rsync-Readonly/ # Set Access Control List permissions for read only
sudo setfacl -m u:rsync-backup:rx /mnt/Backups-Rsync-Readonly/MySQL-Backups
sudo setfacl -m u:rsync-backup:rx /mnt/Backups-Rsync-Readonly/VAR-WWW

Testrun

In order to use these directories, we need to mount the folders. We set the permissions for bindfs and establish the link between the data and our virtual backup folders.

sudo bindfs -o perms=0000:u=rD,force-user=rsync-backup /var/www /mnt/Backups-Rsync-Readonly/VAR-WWW
sudo bindfs -o perms=0000:u=rD,force-user=rsync-backup /Backup/MySQL-Dumps /mnt/Backups-Rsync-Readonly/MySQL-Backups

These commands mount the data directories and create a view. Note that these commands are only valid until you reboot. If the above works and the rsync-backup user can access the folder, you can add the mount points to fstab to automatically mount them at boot time. Unmount the folders before you continue with sudo umount /mnt/Backups-Rsync-Readonly/*  .

Permanently Add the Virtual Folders

You can add the folders to fstab like this:

# Backup bindfs 
/var/www    /mnt/Backups-Rsync-Readonly/VAR-WWW fuse.bindfs perms=0000:u=rD,force-user=rsync-backup 0   0
/Backups/MySQL-Dumps    /mnt/Backups-Rsync-Readonly/MySQL-Backups fuse.bindfs perms=0000:u=rD,force-user=rsync-backup 0   0

Remount the directories with sudo mount -a .

Adding the Keys

In the next step we add the public key from the BackupServer to the authorized_keys file from the rsync-backup user at the WebServer. On the BackupServer, cat the public key and copy the output to the clipboard.

ssh user@backupServer
cat rsync-backup.key.public

Switch to the WebServer and login as rsync-backup user. Then add the key to the file ~/.ssh/authorized_keys.
The file now looks similar like this:

ssh-rsa AAAAB3N ............ fFiUd rsync-backup@webServer```


We then prepend the key with the only command this user should be able to execute: rrsync. We add additional limitations for increasing the security of this account. We can provide an IP address and limit the command execution further. The final file contains the following information:

command=”/usr/bin/rrsync -ro /mnt/Backups-Rsync-Readonly”,from="192.168.0.10”,no-pty,no-agent-forwarding,no-port-forwarding,no-X11-forwarding ssh-rsa AAAAB3N ………… fFiUd rsync-backup@webServer



Now whenever the user rsync-backup connects, the only possible command is rrsync. Rrsync itself is limited to the directory provided and only has read access. We also verify the IP address and restrict the source of the command.

#### Hardening SSH

Additionall we can force the rsync-backup user to use the keybased authentication only. Additionally we set the IP address restriction for all SSH connections in the sshd_config as well.

AllowUsers rsync-backup@192.168.0.10 Match User rsync-backup PasswordAuthentication no



## Backing Up

Last but not least we can run the backup. To start synching we login into the BackupServer and execute the following command. There is no need to provide paths as the only valid path is already defined in the authorized_key file.

rsync -e “ssh -i /home/backup/.ssh/rsync-backup.key.private” -aLP  –chmod=Do+w rsync-backup@webServer: .



# Conclusion

This article covers how a backup user can create backups of data owned by other users without having write access to the data. The backup is transferred securely via SSH and can run unattended. The backup user is restricted to using rrsync only and we included IP address verification. The backup user can only create backups of directories we defined earlier.

Backup your Emails or Migrate between two IMAP Servers with Offlineimap

Syncing IMAP Accounts Between Servers

Due to a change in our mail server infrastructure, I had to move all emails from the old account to a new IMAP account. Obviously if you create the new account in your email client such as thunderbird, you can simply drag and drop the messages between the accounts. Unfortunately, timeouts and other errors prevent you from being sure that all emails have been moved between the servers correctly, if you rely on the mail client only.

Thanks to the tool offlineimap, this task of moving emails can be made much more reliable and it can be automated. The method I describe in this post can also be used for backing up your local machine and preserve your precious emails from being lost, for instance when your provider needs to close down or goes out of business. Offlineimap is a Python project actively developed on Github. Clone the latest version to your local machine as shown below and enter the directory:

git clone https://github.com/OfflineIMAP/offlineimap.git
cd offlineimap

The tool uses a config file where you need to specify your email accounts and settings. In the offlineimap directory, you will find two sample configuration files. The minimal config looks like this:

# Sample minimal config file.  Copy this to ~/.offlineimaprc and edit to
# get started fast.

[general]
accounts = Test

[Account Test]
localrepository = Local
remoterepository = Remote

[Repository Local]
type = Maildir
localfolders = ~/Test

[Repository Remote]
type = IMAP
remotehost = examplehost
remoteuser = jgoerzen

Copy this file to a location (e.g. /home/user/Offlineimap/offlineimaprc_migrate.conf) of your choice and open it in a editor.  For the example, consider that we have two mail servers. The server old.example.org currently contains all the mails that we need to move to the new server new.mailheaven.com. In the terminology of offlineimap, the remote repository is our source old.example.org. We will move the mails to the new server, which is denoted localrepository. This might be a bit confusing, as we plan to move the mails not to our local server, but to the server new.mailheaven.com. So keep this in mind, not to confuse the locations. This is the configuration that we will use in this example. It will copy all emails from the source to the target mailserver over a secure connection.

[general]
accounts = Email-Sync
maxsyncaccounts = 1

[Account Email-Sync]
remoterepository = old_server
localrepository = new_server

[Repository old_server]
type = IMAP
remotehost = old.example.org
remoteuser = olduser@old.example.org
remotepass = secretPassword
ssl = yes
maxconnections = 1
cert_fingerprint = 11111aaaaaaaaaa1111111111cccccccccc111111
folderfilter = lambda folder: folder not in ['Trash','Gesendete Elemente','Notizen','Kontakte','Sent','Junk-E-Mail','Postausgang','Entw&APw-rfe','Kalender','Journal','Gel&APY-schte Elemente','Aufgaben']
readonly = true
[Repository new_server]
type = IMAP
remotehost = new.mailheaven.com
remoteuser = newUser
remotepass = newSecretPassword
ssl = yes
cert_fingerprint = 222222222222222bbbbbbbbb22222222222222222
maxconnections = 1

The config file has several sections. The section general defines which accounts we want to synchronise. Here, we describe an offlineimap account, which is like a task. Do not confuse it with your actual email accounts. They are called repositories in offlineimap speech. In our example we have only one account called Email-Sync. The next section defines what we want to do with this account. We specify the source and the destination email accounts. As already said, the actual email accounts are denoted as repositories. Each repository is defined in its own section. This is the place to define the actual email accounts. Make sure to activate SSL. If offlineimap should complain about the CA or the certificate of the mail servers. it is very likely that you use self signed certificates and you need to tell offline imap that this is a trusted domain. We can achieve this by providing the fingerprint of the mail server certificate. You can use the following command to get the fingerprint from the mail server:

openssl s_client -connect example.org:993 < /dev/null 2>/dev/null | openssl x509 -fingerprint -noout -in /dev/stdin | sed 's/./\L&/g' |  sed 's/\://g'```


This snippet retrieves the public certificate of the mail server and prints it in the terminal. The gmail mail server for instance has this fingerprint:

 openssl s_client -connect imap.gmail.com:993 < /dev/null 2>/dev/null | openssl x509 -fingerprint -noout -in /dev/stdin SHA1 Fingerprint=E1:D6:E8:F1:69:72:75:CF:8F:1C:CF:E6:35:B0:42:30:C0:1D:62:C2```

Offlineimap expects the fingerprint in lower case letters and without the colon. The command above turns the fingerprint string into lower case and removes the colons. So again with the Gmail example, this is what the command does:

openssl s_client -connect imap.gmail.com:993 < /dev/null 2>/dev/null | openssl x509 -fingerprint -noout -in /dev/stdin | sed 's/./\L&/g' |  sed 's/\://g'
sha1 fingerprint=e1d6e8f1697275cf8f1ccfe635b04230c01d62c2

You just need the later part of the line and copy it into the config file. Note that the configuration above stores your passwords in plain text, which is in general a bad idea. If you remove the lines with remotepass from the config file, offlineimap will prompt you for your passwords.

Create a Local Backup

Offlineimap is also suited very well for creating local backups of your IMAP accounts. You can store all your emails in the maildir format, which creates a folder structure with your messages on the hard drive. Use a config file similar file like the one below:

[general]
accounts = Mail-Server

[Account Mailserver-Local]
localrepository = Local-Backup
remoterepository = Mail-Server

[Repository Local-Backup]
type = Maildir
localfolders = /media/BackupPartition/Offline-Imap_BACKUP/
sep = /

[Repository Mail-Server]
type = IMAP
remotehost = mailserver.at
remoteuser = stefan
ssl = yes
maxconnections = 2
cert_fingerprint=33cccccccccccccccddddddddddddd4444444444
realdelete = no
readonly=true
folderfilter = lambda folder: folder not in ['Trash','Gesendete Elemente','Notizen','Kontakte','Junk-E-Mail','Postausgang','Entw&APw-rfe','Kalender','Journal','Gel&APY-schte Elemente','Aufgaben']

The realdelete=no flat option prevents that emails which have been deleted from your mail server will also be removed from the local backup.

Start Syncing

After you have your configuration ready, you can test the tool with the option –dry-run like this:

./offlineimap.py -c /home/user/Offlineimap/offlineimaprc_migrate.conf

Unfortunately, offlineimap throws errors in the dry-run, which is does not in the actual run. So if you can connect, you may need to run it directly. Note that we included the readonly flag for our source repository, in order to prevent changes there. Offlineimap can be a bit picky about folder names and translation rules. In the example above we excluded some folders from the transfer, as they caused problems either because they contained umlauts or other issues. Also there seems to be a race condition sometimes, leading to offlineimap complaining about existing folders. Repeating the process of creating the folders and waiting a few seconds seemed to resolve the problem. The documentation of offlineimap is quite extensive and a great source for finding out more on that topic.