I manage the cloud server and the web applications of an organization. These applications are mostly dockerized and installed by docker-scripts. For robustness, I would like other people from the organization to be able to keep a backup of whatever is installed on the server, but without being able to restore or modify anything on the server (they can restore it on a new server, if needed). Using rsync is one of the obvious choices for making such a backup, but it also needs a couple of tricks for working properly, the way that I want it. In this blog I describe this setup.

First of all we need to make sure that rsync is installed both on the client and on the server:

apt install rsync

1. Create a backup user, for example backup1 (in ubuntu there is already a user named backup):

   useradd backup1 -m
   ls -al /home/backup1/
   

2. Create a ssh key-pair for this user:

   ssh-keygen -t ecdsa -P '' -q -f key1
   
   ls -l key1*
   cat key1
   cat key1.pub
   

3. Add the public key to /home/backup1/.ssh/authorized_keys:

   mkdir -p /home/backup1/.ssh
   chown backup1: /home/backup1/.ssh
   chmod 700 /home/backup1/.ssh
   
   cat key1.pub >> /home/backup1/.ssh/authorized_keys
   chown backup1: /home/backup1/.ssh/authorized_keys
   chmod 600 /home/backup1/.ssh/authorized_keys
   
   ls -al /home/backup1/.ssh/
   cat /home/backup1/.ssh/authorized_keys
   

4. Try to login with this key:

   ssh -p 22 -i key1 backup1@localhost
   

   You should be able to login without a password.

5. Try to copy something:

   mkdir -p /home/backup1/test1
   touch /home/backup1/test1/file1.txt
   touch /home/backup1/test1/file2.txt
   ls -al /home/backup1/test1
   
   rsync -a -e "ssh -p 22 -i key1" backup1@localhost:~/test1 .
   
   ls -al test1
   

1. Let's find out the command that the client is sending to the server through SSH. Let's try the same rsync command again, with the added SSH switch -v (verbose):

   rsync -a -e "ssh -p 22 -i key1 -v" backup1@localhost:~/test1 .
   

   Then let's look for the debug line that says "Sending command":

   rsync -a -e "ssh -p 22 -i key1 -v" backup1@localhost:~/test1 . 2>&1 \
       | grep "Sending command"
   

   It should be something like this:

   rsync --server --sender -logDtpre.iLsfxC . ~/test1
   

2. We can restrict the SSH key key1 to execute only this command and nothing else. For this we need to add something like this before the public key on /home/backup1/.ssh/authorized_keys:

   command="rsync --server --sender -logDtpre.iLsfxC . ~/test1" ecdsa-sha2-nistp256 AAAAE2Vj....
   

   To make it even more secure, we can also add the options no-agent-forwarding,no-port-forwarding,no-pty,no-user-rc,no-X11-forwarding. The file /home/backup1/.ssh/authorized_keys now should look like this:

   command="rsync --server --sender -logDtpre.iLsfxC . ~/test1",no-agent-forwarding,no-port-forwarding,no-pty,no-user-rc,no-X11-forwarding ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMbMdR9uW4SMeinpVvr6UQZaFybkiVZxm2DRYxFlCuxHchpTMGR7U4gZGZwY4D5LQDDy1Py4TWSsEizda4LecgQ= root@server
   

3. Let's check that now we cannot login with key1 anymore, but we can still use it to rsync:

   ssh -p 22 -i key1 backup1@localhost  # should fail
   
   rm -rf test1
   rsync -a -e "ssh -p 22 -i key1" backup1@localhost:~/test1 .
   ls -l test1
   
   rm -rf test1
   rsync -a -e "ssh -p 22 -i key1" backup1@localhost: .
   ls -l test1
   

1. Install bindfs:

   apt list bindfs
   apt show bindfs
   apt install bindfs
   

2. Create mount directories:

   mkdir -p /mnt/backup-server/scripts
   mkdir -p /mnt/backup-server/apps
   

3. Add these lines to /etc/fstab for mounting directories read-only:

   /opt/docker-scripts /mnt/backup-server/scripts fuse.bindfs perms=0000:u=rD,force-user=backup1,force-group=nogroup 0 0
   /var/ds             /mnt/backup-server/apps    fuse.bindfs perms=0000:u=rD,force-user=backup1,force-group=nogroup 0 0
   

   Since we are using docker-scripts for installing and managing apps, these two directories are what we need to backup: /opt/docker-scripts and /var/ds.

4. Mount them:

   mount -a
   ls -al /mnt/backup-server/scripts
   ls -al /mnt/backup-server/apps
   

5. Test that they are read-only:

   sudo -u backup1 ls -al /mnt/backup-server/scripts
   sudo -u backup1 touch /mnt/backup-server/scripts/test1.txt
   

1. For convenience, we can combine the command and the key in a bash script named backup-server.sh that looks like this:

   #!/bin/bash
   
   server=127.0.0.1
   port=22
   
   keyfile=$(mktemp)
   sed -n -e '/^----/,/^-----/p' $0 > $keyfile
   
   cd $(dirname $0)
   rsync -a -e "ssh -p $port -i $keyfile" backup1@${server}: .
   
   rm -f $keyfile
   exit 0
   
   -----BEGIN OPENSSH PRIVATE KEY-----
   MHcCAQEEIGHMv+mlPZg/V6TmLADsfZcpMRcfxykJGzXAF65tuQ4AoAoGCCqGSM49
   AwEHoUQDQgAExsx1H25bhIx6KelW+vpRBloXJuSJVnGbYNFjEWUK7EdyGlMwZHtT
   iBkZnBjgPktAMPLU/LhNZKwSLN1rgt5yBA==
   -----END OPENSSH PRIVATE KEY-----
   

   Let's try it:

   chmod 700 backup-server.sh
   rm -rf test1/
   ./backup-server.sh
   ls -l test1/
   

2. Now we can move this script to the client (backup server), making sure to set the proper values for the variables server and port, and it should work.
3. Let's also fix the directory on the server that is being backed up. We should edit /home/backup1/.ssh/authorized_keys and change ~/test1 to /mnt/backup-server

4. On the client (computer that is receiving the backup), let's place the script backup-server.sh on a directory like /var/backup:

   mkdir -p /var/backup
   mv backup-server.sh /var/backup/
   
   cd /var/backup/
   ./backup-server.sh
   

5. Let's also create a cron job that runs this script periodically each week:

   cat <<EOF > /etc/cron.d/backup-server
   # backup the server each tuesday
   0 0 * * TUE  root  /var/backup/backup-server.sh
   EOF