I put out a request today to my followers Twitter asking this question:

I've had the following services recommended to me by a number of fellow developers whose opinions I have a lot of respect for:

A couple of ex-Headshifters pointed me to Duplicity, a free tool that also looks very promising. Using it looks pretty straight forward.

    duplicity /home/my_directory scp://backup@other.host//usr/backup

A variant of it can be used to backup to Amazon S3 too, and there's a helpful blog post to show how to do this here, if you use Ubuntu or OSX.

Kalv recommended Safe, a clever Ruby gem by the Astrails crew that was originally designed to automate the backing up of Rails projects in a simple fashion.

One service that looks really interesting is though is Tarsnap, as recommended by [Jon Gilbrath][]. It's similarly simple to use, but takes care of the some of the more awkwards backup issues, and saves you having to setup your own S3 account.

  # Create an archive named "mybackup" containing /usr/home and /other/stuff:
  tarsnap -c -f mybackup /usr/home /other/stuff

The developer, Colin Percival also has written quite extensively about how it works on his own blog - he's only making a few cents per gigabyte on providing this service for people, yet it still looks to be something viable for him to run - amazing.

It looks like it's almost exactly what I'm after - the only thing missing is the option to back up storage inside the EU. This requirement is mainly one coming from data protection concerns from previous clients, because rules for processing data and storing in in the EU are different to that in the US, but given the strength of encryption, I'm not sure how much of an issue this really is, these days.

I'd really appreciate some light shed on this actually - technology is moving so much faster than law these days, it's frustrating not being able to take advantage of these kinds of services.

Anyway that's what I've found. What do you use, and why?

This meant I had to use the zip tool, which I can never remember how to use, hence this memory jogging post.

Now if someone's using a Mac or is on Linux, the normal command line approach to to tar up a directory to send to someone goes like this:

tar -cvzf my_tarfile.tar.gz a_directory_to_tar and_otherone_here

We're calling tar here, with the -create (c), file (f), compress (z) oh, and the verbose (f), flag. Think of that string like saying to the computer: "create the tarfile my_tarfile putting it into a file we just created, verbosely, while compressing it, from a_directory_to_tar and and_otherone_here".

As commands go, the order feels a bit weird at first, but you get the hang eventually.

Where zip is different

Using zip isn't too different really, but it you need to remember to tell it to zip recursively (why is this not default behaviour?). If we call command:

zip archive a_directory_to_tar and_otherone_here

We'll end up with an rather useless empty file called archive.zip. We need to pass in the recursive flag

zip -r archive a_directory_to_tar and_otherone_here

There's also no need to add the .zip file extension - this is added for you as a courtesy. Well, after that ridiculous non-recursive default shenanigans, frankly that's the least it could do...

Anyway, there you have it. The differences you between them if you don't want to dive into the manpages just to package up a folder for somebody, or look at this linux.about.com.

... and as soon as I say this, the database powering this site corrupts on me - charming!

For one awful moment I thought I had lost everything (I hadn't set up a regular mysql backup task for this blog), but thankfully, bringing this blog back from the dead was surprisingly straight forward.

Here's what happened, and what I did to fix it.

Yesterday when writing a post about how there's a mobile missing tariff that should exist but doesn't, when I posted, WordPress would accept my post; those pretty git spinners just kept spinning on the text page.

I didn't think too much of it, at the time, as I was heading out to meet James to work on some top sekrit project, based around node.js, mongodb, and doing something strange yet hopefully useful with wifi, so I just saved it locally on my mac, and headed out on my bike.

This evening though, I tried again, and I had the same error. "Ah," I thought, "maybe I just need to restart the database on this box - it's been going for ages anyway". So I ssh'd into the server running the system, and called the usual CentOS restart command

  service mysql restart

This took nearly two minutes to stop, then spat out this response when restarting:

[chris@stemcaa2 ~]# service mysql restart
Shutting down MySQL........................................[  OK  ].......
Starting MySQL.........................................................................................................................................................................................................................................../sbin/service: line 66: 27651 Terminated              env -i LANG="$LANG" PATH="$PATH" TERM="$TERM" "${SERVICEDIR}/${SERVICE}" ${OPTIONS}

Okay, slight panic now - lets check the disk, to see if there's space for the database to write, with df -h

 
[chris@stemcaa2 ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1             9.9G  9.4G     0 100% /
tmpfs                 129M     0  129M   0% /dev/shm
/usr/tmpDSK           485M   11M  449M   3% /tmp

Ah, that's not good. Disk space is like oxygen for servers, and without it, things break quickly.

The quickest way to make some breathing space fast is to clear the yum cache, like so:

yum clean headers packages

Okay, lets try restart again:

[chris@stemcaa2 ~]# service mysql restart
Shutting down MySQL.                                       [  OK  ]
Starting MySQL.                                            [  OK  ]

Sweet! It worked! But hang on... now wordpress isn't showing any posts! This really isn't good.

So lets check the database;

[root@stemcaa2 ~]# mysql
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 11
Server version: 5.0.90-community MySQL Community Edition (GPL)
 
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
 
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema | 
| antia_wordpress    | 
| chris_wordpress    | 
| cphulkd            | 
| deadonim_wordpress | 
| dov_wordpress      | 
| eximstats          | 
| horde              | 
| leechprotect       | 
| modsec             | 
| mysql              | 
| roundcube          | 
+--------------------+
12 rows in set (0.16 sec)

Okay the database is there. What about the tables though?

 
Database changed
mysql> show tables;
+---------------------------+
| Tables_in_chris_wordpress |
+---------------------------+
| wp_ak_twitter             | 
| wp_commentmeta            | 
| wp_comments               | 
| wp_links                  | 
| wp_options                | 
| wp_postmeta               | 
| wp_posts                  | 
| wp_term_relationships     | 
| wp_term_taxonomy          | 
| wp_terms                  | 
| wp_usermeta               | 
| wp_users                  | 
+---------------------------+
12 rows in set (0.00 sec)

Hmm.. they're there too. Maybe the tables?

 
mysql> select * from wp_posts
    -> ;
ERROR 145 (HY000): Table './chris_wordpress/wp_posts' is marked as crashed and should be repaired

Uh-oh. I think this is going to be ugly. And yet...

The magic fix

...a quick check here on google gleaned the fix - MySQL's got a tool for just this occasion, and you call it like mysqlcheck broken_database from the command line:

[root@stemcaa2 ~]# mysqlcheck chris_wordpress
chris_wordpress.wp_ak_twitter                      OK
chris_wordpress.wp_commentmeta                     OK
chris_wordpress.wp_comments                        OK
chris_wordpress.wp_links                           OK
chris_wordpress.wp_options                         OK
chris_wordpress.wp_postmeta                        OK
chris_wordpress.wp_posts
warning  : Table is marked as crashed
error    : Size of datafile is: 745472         Should be: 745692
error    : Corrupt
chris_wordpress.wp_term_relationships              OK
chris_wordpress.wp_term_taxonomy                   OK
chris_wordpress.wp_terms                           OK
chris_wordpress.wp_usermeta                        OK
chris_wordpress.wp_users                           OK

The fix was trivial from here - just pass in an auto-repair flag:

 
[root@stemcaa2 ~]# mysqlcheck chris_wordpress --auto-repair
chris_wordpress.wp_ak_twitter                      OK
chris_wordpress.wp_commentmeta                     OK
chris_wordpress.wp_comments                        OK
chris_wordpress.wp_links                           OK
chris_wordpress.wp_options                         OK
chris_wordpress.wp_postmeta                        OK
chris_wordpress.wp_posts
warning  : Table is marked as crashed
error    : Size of datafile is: 745472         Should be: 745692
error    : Corrupt
chris_wordpress.wp_term_relationships              OK
chris_wordpress.wp_term_taxonomy                   OK
chris_wordpress.wp_terms                           OK
chris_wordpress.wp_usermeta                        OK
chris_wordpress.wp_users                           OK
 
Repairing tables
chris_wordpress.wp_posts
info     : Found block that points outside data file at 742932
status   : OK

And then all was well again! Why can't technology always be this easy to fix? A huge, huge heartfelt thanks goes to Felipe Cruz for adding such simple instructions on his site explaining how to use that insanely handy mysqlcheck command.

I had fun discovering the difference between the linux "killall" command (kills all processes matching the specified name, useful for stopping zombies) and the solaris "killall" command (kills all processes and halts the system, useful for stopping the production server in the middle of peak hours and getting all your co-workers to laugh at you for a week)

Remind me to avoid looking after any Solaris boxes at all costs from now on - if a fairly innocuous Linux command kills an entire system, I shudder to think what other pitfalls and pratfalls lie in wait for a poor sysadmin....

However, not every webhost automatically has accounts setup to allow for this, and if click the promising looking auto-upgrade icon, you’ll often be face with this screen here:

You get this screen when the folder containing wordpress belongs to the user, instead of the apache daemon; this is normally either www-data if you’re using Ubuntu, or nobody if you’re using Redhat Linux instead.

The simplest way to do this is to change the folders to be owned by the apache daemon instead, by using chmod like so:

chown -R nobody wordpress

Adding the -R flag makes the changing of ownership recursive through all the folders inside wordpress.

Job done.

This is outlined in more detail by Rob Spencer on his own blog – really quite handy.

This year, as I use web app frameworks like Django or Rails more and more, I've increasingly been using the command line instead of tools like Transmit for making changes to sites and web apps. This works fine locally, but when you start using remote servers, typing passwords all day gets old quickly.

This is where logging into servers using ssh keys comes in. This a very high level abstraction, and isn't that accurate technically, I found it easiest to think of this process as three separate steps:

1) Generate a lock and key
2) Put the lock on the server.
3) Use the key to get into the server.

Generate the lock and key - generate your key pair

Essentially what you do is generate a private/public key pair, and put the public part of the key onto the remote server. The server can then check your the public key you gave it against the private key on your laptop, and if they 'fit', it's pretty sure you are who you say you are.

First , you generate the key pair, using the program ssh-keygen, and telling it to use the encryption type 'rsa':

ssh-keygen -t rsa

The two common options here are to use -rsa type encryption, or -dsa encryption. DSA stands for Digital Signature Algorithm, whereas RSA is named after the three inventors of the algorithm; Ron Rivest, Adi Shamir and Leonard Adleman.

I haven't found any real difference that affects my experience of ssh as a developer, but the gist I get from skimming the newsgroups is that DSA is a slightly faster encryption algorithm than RSA, but also slightly less difficult to (eventually) crack.

My understanding earlier was that DSA was the more 'free' encryption method, as RSA was under patent initially, but a quick check on wikipedia tells us that RSA released to the public domain in the year 2000, so for most of us non-crypto profs, there's almost nothing in choosing between the two now.

Running this command will create a public/private key pair, usually put in an invisible directory called .ssh, and they should look like this if you peep inside the ~/.ssh directory:

id_rsa

Your private key, which you keep on your computer. This shouldn't be shared with anyone, and though it's not totally accurate, I like to think of ssh key pairs in similar terms to locks and keys. In this case we can think of this as the front door key.

id_rsa.pub

your public key which you put on the remote server, specifically in a file called authorized_keys. You can think of this as the 'lock' that you would put the id_rsa private key into.

Put your the lock (the public key) on the remote server

You next need to put the public key into the hidden ~/.ssh directory, appending the public key to a file called authorized_keys on the remote server. When you next try to ssh into your server, you'll end up combining your private key on your computer, with the public key on the server, to give a fairly strong claim to you being who you say you are.

Once you've copied it onto your server, which you can do using an sftp client, or simply using scp,You need to make the server aware of the public key by writing it to a text file called authorized_keys. A simple way to do this is to use a command called cat (think concatenate):

cat .id_rsa.pub >> ./ssh/authorized_keys

This takes the contents of your public key, and writes it to the end of the authorized keys file.

To continue the clumsy lock and key analogy, this is like fitting the front door with the new lock that responds only to your private key.

Exit the ssh session, and then try logging in once more..

Log in

Et voila! You should now have passwordless log-in to a server setup from your computer to a remote server now. You can use a variant of this setup to create secure automated backups from one server to another, and to set up other dull tasks that are best automated.

It doesn't stop here though - you can ramp up security here by limiting what tasks a script, or person logging in using ssh can do on a server, and you can add extra layers of security to make key pairs AND passwords needed for certain accounts, or certain actions, or set up elaborate daisy chain sequences where an action on one computer sets of a series of other automated tasks on other computers around the web that you'd previously have to do by yourself. But that's for a future post.

Common gotchas here

I've lost a depressing amount of time trying to setup ssh keys for passwordless login, before realising that I needed to fiddle with file permissions on the relevant files.
If you're still being challenged for a password when you're signing, check that you have the permissions setup properly on your server. You need to be the only person able to alter the authorized_keys file, on the remote server, and you can do this with the tool chmod, like so:

chmod 600 ~/.ssh/authorized_keys

This gives you read and write access to the authorized_keys file (the 6), but removes anyone else's access to it (the second and third 0's). Which makes sense really - for security you want to be the only person who can decide who is allowed to add authorised user's public keys.

You also need to check that your hidden ~/.ssh has similar permissions: you want to be the only person who can edit the directory, so make sure this is reflected in the permissions. You need to set chmod to 700:

chmod 700 ~/.ssh/

Once again, this grants access only to you, and no one else.

Isn't logging into a server without a password a horrifically bad idea security wise? What if someone steals your laptop?

If you lose the keys to your house, it's a good idea to change the locks, so people with those keys can't get in. Likewise, if you lose your computer, the equivalent action to changing your locks would be removing that laptop's public key from your server's list of authorised keys.

I've borrowed heavily from James Gardner's own writing about ssh, so if you're interested in learning more, he's written quite lucidly and at length about them on his own blog.