Archive for the ‘backup’ Category

My Server

16, March 2011

Laptop Server
The Server I use is an Aspire Laptop, 5633WLM Core 2 Duo T5500 1.66 GHz with 1Gb of RAM with a missing screen [don’t ask] with a standard install of Xubuntu. I’ve built a rack so that it stands upright so save space and it is linked to my monitor via a KVM switch. It runs an Apache web server, an Apt cache server, backup server to an attached USB Hard Drive, the Folding@Home application, Torrent Server, a bunch of other cron run scripts and the Mediatomb Media Server.

Home Server

Home Server

Apt Server
Apache Server
Folding@Home
Torrent Server

Backup using rsync, SSH & cron

31, August 2009

Backup on Ubuntu.

Updated 09/11/09 to correct some errors!

This “How To” was written with the help of this article by Troy Johnson, here and amended were necessary to fit my needs.

I have 2 PC’s on a network and want to backup some files from one PC the [remote_client] to the other PC the [server] to do this I want to run a script from the server. The process can be achieved using rsync & SSH and automated with a cron job on the server.

Step:1

We install on the server

open-ssh-server and rsync

and on the client openssh-client and rsync

This can be done via the Synaptic Package Manager or using the command line i.e.

~$ sudo apt-get install openssh-client

~$ sudo apt-get install rsync

you also may want to check that cron is installed and if you want a GTK version of rsync you could install gadmin-rsync.

So the files I want to backup are on the remote_client in the following folder; /home/martin/my_files

and I want to back them up on the server in the following folder; /home/martin/backups

Note: I have an account on both PC’s with the same name and password, and read access to both the source & destination folders.

On the server we test out a basic script called backup.sh and saved with the Permissions set to “allow executing as a program”, check out my post here for help with that, the content of the script is;

#!/bin/sh

## first script to copy files from remote_client to server ##

rsync -avz -e ssh martin@remote_client:/home/martin/my_files/ /home/martin/backups

save the script and then run it by typing;

~$ ./backup.sh

in a Terminal assuming you saved the script in your home directory /home/martin and called it backup.sh. You should be prompted for your user password on the remote_client then the process should start.

Step:2

Ok so that does the trick but if I’m to automate this script it needs to work without SSH asking me for my password each time. To do this I need to generate a private/public key pair, this is much more secure than adding the actual password in the text of the script or elsewhere linked to it.

We now need to be logged onto the server;

To generate the key pair I open a [Terminal] and issue the command;

~$ mkdir /home/martin/.key

to create a hidden folder in my home folder called .key [the dot makes it hidden], then;

~$ ssh-keygen -t dsa -b 1024 -f /home/martin/.key/server-rsync-key

Generating public/private dsa key pair.

Enter passphrase (empty for no passphrase): [press Enter]

Enter same passphrase again: [press Enter]

Your identification has been saved in /home/martin/.key/server-rsync-key.

Your public key has been saved in /home/martin/.key/server-rsync-key.pub.

The key fingerprint is:

etc. etc.

I now have 2 files in the .key folder in my home folder called;

server-rsync-key

and

server-rsync-key.pub

As you will later set-up a cron job which will run a script that needs access to the junk-rsync-key file we need to change the default ubuntu file permissions to owner, read/write group & other to none

$ chmod 600 server-rsync-key

Now copy the other key, server-rsync-key.pub over to the remote_client into my home folder /home/martin and log onto the remote_client.

Now on the remote_client, I issue the following command in a [Terminal]

~$ if [ ! -d .ssh ]; then mkdir .ssh ; chmod 700 .ssh ; fi

[this checks to see if the folder .ssh exists and if not creates it]

~$ mv server-rsync-key.pub .ssh/

[this moves the key from my home folder to the .ssh folder, hidden in my home folder]

~$ cd .ssh/

[we now move to the .ssh folder]

~$ if [ ! -f authorized_keys ]; then touch authorized_keys ; chmod 600 authorized_keys ; fi

[this checks to see if there is an authorized_keys file and if not creates it with the correct permissions]

~$ cat server-rsync-key.pub >> authorized_keys

[this copies the contents of server-rsync-key.pub key into the file authorized_keys file.

If we now run our original script with the amendment below on the server it no longer asks for a password.

rsync -avz -e “ssh -i /home/martin/.key/server-rsync-key” martin@remote_client:/home/martin/my_files/ /home/martin/backups

Step:3

Now the key can be used to make connections to the remote_client, but these connections can be from anywhere (that the ssh daemon on remote_client allows connections from) and they can do anything (that remote_user can do), which could be dangerous.  To make this more secure edit the ‘authorized_keys’ file in remote_client /home/martin/.ssh folder with a text editor and modify the line ending user_name@server_name from this:

ssh-dss
yl6b2/cMmBVWO39lWAjcsKK/zEdJbrOdt/sKsxIK1/ZIvtl92DLlMhci5c4tBjCODey4yjLhApjWgvX9
D5OPp89qhah4zu509uNX7uH58Zw/+m6ZOLHN28mV5KLUl7FTL2KZ583KrcWkUA0Id4ptUa9CAkcqn/gW
kHMptgVwaZKlqZ+QtEa0V2IwUDWS097p3SlLvozw46+ucWxwTJttCHLzUmNN7w1cIv0w/OHh5IGh+wWj
V9pbO0VT3/r2jxkzqksKOYAb5CYzSNRyEwp+NIKrY+aJz7myu4Unn9de4cYsuXoAB6FQ5I8AAAEBAJSm
DndXJCm7G66qdu3ElsLT0Jlz/es9F27r+xrg5pZ5GjfBCRvHNo2DF4YW9MKdUQiv+ILMY8OISduTeu32
nyA7 etc....

to this:

from="10.1.1.1", command="/home/martin/.key/validate-rsync" ssh-dss bA402VuCsOLg
yl6b2/cMmBVWO39lWAjcsKK/zEdJbrOdt/sKsxIK1/ZIvtl92DLlMhci5c4tBjCODey4yjLhApjWgvX9
D5OPp89qhah4zu509uNX7uH58Zw/+m6ZOLHN28mV5KLUl7FTL2KZ583KrcWkUA0Id4ptUa9CAkcqn/gW
kHMptgVwaZKlqZ+QtEa0V2IwUDWS097p3SlLvozw46+ucWxwTJttCHLzUmNN7w1cIv0w/OHh5IGh+wWj
V9pbO0VT3/r2jxkzqksKOYAb5CYzSNRyEwp+NIKrY+aJz7myu4Unn9de4cYsuXoAB6FQ5I8AAAEBAJSm
DndXJCm7G66qdu3ElsLT0Jlz/es9F27r+xrg5pZ5GjfBCRvHNo2DF4YW9MKdUQiv+ILMY8OISduTeu32
nyA7 etc....

where “10.1.1.1” is the IP address of server, and “/home/martin/.key/validate-rsync” point to a script called validate-rsync as follows:

#!/bin/sh
case “$SSH_ORIGINAL_COMMAND” in
*\&*)
echo “Rejected”
;;
*\(*)
echo “Rejected”
;;
*\{*)
echo “Rejected”
;;
*\;*)
echo “Rejected”
;;
*\<*)
echo “Rejected”
;;
*\`*)
echo “Rejected”
;;
*\|*)
echo “Rejected”
;;
rsync\ –server*)
$SSH_ORIGINAL_COMMAND
;;
*)
echo “Rejected”
;;
esac

If server has a variable address, or shares its address (via NAT etc.) with hosts you do not trust, omit the ‘from=”10.1.1.1″,’ part of the line (including the comma), but leave the ‘command’ portion. This way, only the ‘rsync’ will be possible from connections using this key. Make certain that the ‘validate-rsync’ script is executable by remote_user on remote_client and test it.

So in my case the result looks like this:

command="/home/martin/.key/validate-rsync" ssh-dss bA402VuCsOLg9YS0NKxugT+o4UuIj
yl6b2/cMmBVWO39lWAjcsKK/zEdJbrOdt/sKsxIK1/ZIvtl92DLlMhci5c4tBjCODey4yjLhApjWgvX9
D5OPp89qhah4zu509uNX7uH58Zw/+m6ZOLHN28mV5KLUl7FTL2KZ583KrcWkUA0Id4ptUa9CAkcqn/gW
kHMptgVwaZKlqZ+QtEa0V2IwUDWS097p3SlLvozw46+ucWxwTJttCHLzUmNN7w1cIv0w/OHh5IGh+wWj
V9pbO0VT3/r2jxkzqksKOYAb5CYzSNRyEwp+NIKrY+aJz7myu4Unn9de4cYsuXoAB6FQ5I8AAAEBAJSm
DndXJCm7G66qdu3ElsLT0Jlz/es9F27r+xrg5pZ5GjfBCRvHNo2DF4YW9MKdUQiv+ILMY8OISduTeu32
nyA7 etc....

Our backup.sh script now looks like this:

#!/bin/sh

## second script to copy files from remote_client to server ##

rsync -avz -e “ssh -i /home/martin/.key/server-rsync-key” martin@remote_client:/home/martin/my_files/ /home/martin/backups

Step:4

I now need to run the script from a Cron job, first move the backup.sh script into your $PATH, for help with this check out my post here.

Now on the server open a [Terminal] and issue the command:

~$ crontab -e

This should open a text editor, normally vi, normally paste into it the following text as a reminder

# leave a space between each option #
# the * is a wildcard option #
# dayofmonth is 1 to 31 #
# dayofweek can be 0 – 6, 0 is Sunday #
# or use mon tue wed etc #
#
# mins hours dayofmonth month dayofweek command
# * * * * * * command

Then use Crontrol+o to save and press [Enter] to save, then Control+x to [Exit].

and test…