sudo apachectl restart

This isn't a really destructive command, and having to type in your admin password every time when doing this in development on your own computer gets old quickly. It's also error prone. Surely there's a better way?

Fortunately, when browsing the Aegir OS X install documentation, I came across as handy fix to this problem. The Aegir hackers let Aegir handle server restarts in a fairly elegant fashion, by tweaking the sudoers file on your mac, which is basically a short list of who is allowed to do what on your machine. I've borrowed a few tricks, and adaprted them to use in my sudoers file here, and after showing it in full, I'll explain how it works.

Bear in mind, you can't edit the sudoers file directly - you need to use the visudo command, (this works as a precaution to stop this file getting screwed up by letting more than one person is edit it at a time for example).

Also, to make things more complex, you need to edit this inside the terminal, to you may need to force this by typing EDITOR='vim' first

Okay, now that's out the way, lets look at that file:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

  # Run as alias specification
 
  # User privilege specification
  root    ALL=(ALL) ALL
  %admin  ALL=(ALL) ALL
 
  # Uncomment to allow people in group wheel to run all commands
  # %wheel        ALL=(ALL) ALL
 
  # Same thing without a password
  # %wheel        ALL=(ALL) NOPASSWD: ALL
  %staff          ALL=(ALL) NOPASSWD: /usr/sbin/apachectl
  # Samples
  # %users  ALL=/sbin/mount /cdrom,/sbin/umount /cdrom
  # %users  localhost=/sbin/shutdown -h now

Lets look at the first lines, with root and %wheel. If you're even bothering to read this, the chances are you know that root refers to the all powerful user that can do anything on a system, but you may not be familiar with the percent prefix on %admin nor the ALL=(ALL) ALL. The %admin basically means 'anyone in the admin group, but the ALL=(ALL) ALL is somewhat more cryptic. The rough translation goes like this though:

from ALL terminals, let these users run ALL commands and as ALL of the users in the system.

We see the same trick visible again with the %wheel group, but the line starting with %staff deserves more attention:

  %staff          ALL=(ALL) NOPASSWD: /usr/sbin/apachectl

Translated, this means:

for ALL members in the staff group, let them use ALL terminals, to run the command /usr/sbin/apachectl as ALL users (in particular, the root user) without needing a password (that's the NOPASSWD: bit).

This is the line that lets us run the familiar sudo apachectl restart without needing to constantly type our password credentials in.

Which over the course of a year, will easily save you tons of typing over the year, and leave some time to skim the sudoers man page, and suggest a similar trick here for others to try.

Over to you now...

Assuming you've access to the command line and drush, you can pull a similar trick with Drupal, by typing the following query in:

    drush sql-query "update users set pass=md5('NEWPASSWORD') where uid = 1;"

What's happening here?

The first thing we're doing is calling drush sql-query, a sub-command of drush.

If you haven't used Drush yet, you really, really should. It totally transforms how you work with Drupal, by making the kinds of tasks you had to do manually through the website possible from a commandline, which means yes, you can get up to all kinds of handy scripting shenanigans.

As you might expect drush sql-query lets you pass a single arbitrary query to the database described in your site's settings file, without you needing to fish the credentials out yourself. Here's our query too now:

 update users set pass=md5('NEWPASSWORD') where uid = 1;

In short, we're updating the users table in the Drupal database, by setting the pass_word value for the the first user id (_where uid=1), to a md5 hash of the phrase NEWPASSWORD.

If you don't haves access to drush, nor the command line, but you still can change a file over SFTP, you can to the same by adding a snippet like this on to the site:

  $doh_forgot_my_password = db_query("update users set pass=md5('NEWPASSWORD') where uid = 1;");

Of course you really should be paramaterizing this like so:

  $doh_forgot_my_password = db_query("update users set pass=md5('%s') where uid = %d;", array("NEWPASSWORD", "1") );

But this given the fact that this snippet should only existing in a template for 15-20 seconds at the most you'd probably be forgiven for taking the short cut...

I had no idea could pull ddebug_backtrace() to get an instant stacktrace, or DARGS() to see the arguments being passing into a function at any point, or DD() to log directly to a text file (which you can tail in a terminal window to see stuff as it happens).

Now, if only Simpletest wasn't so depressingly slow...

It's pretty handy - it allows you to step your code when things break, giving you an idea of what exactly is happening, under the hood, or letting you understand the actual a request takes through your code.

I'm not using MAMP, because where possible, I'm trying to minimise duplication of software on the my computer, and previously I've confused myself something horrible with trying to keep track of multiple instances of MySQL and Apache before, so if you're not using MAMP, following these instructions should help get you set up with a decent debugging tool, on Snow Leopard, and along the way get a fairly maintainable lamp stack, if you're not enamoured with MAMP.

Get PHP 5.2

This isn't strictly necessary, but right now, I've found PHP 5.2 to be less hassle when developing than 5.3, and for time being, Drupal and WordPress to be standardising on it first before moving to the newer version, which means I am too.

The best instructions I've found to do this are here PHP 5.2 for compatibility with Drupal, and provide a good introduction to the amazing Homebrew, which you really should be using instead of Macports or Fink, if you're not already.

Lets get the hard commandline work out of the way first.

Setup PEAR to complement Homebrew's PHP 5.2

Just like how we have pip or easy_install with Python, CPAN for Perl, and Rubygems for Ruby, with PHP, we have PECL, and PEAR. PEAR is repository of PHP classes, like PHP Unit for unit testing, whereas PECL is a repository for C extensions like APC or Memcache, that help with performance, and caching, or link to other programs.

By default we do have PEAR and PECL installed, but in keeping with Homebrew's example of storing stuff in /usr/local/ to avoid needless sudo'ing we're going to use our own versions of PEAR, using this handy one liner. What's happening here is that we're using curl to fetch the data at http://pear.php.net/go-pear, and then streaming it into the php command:

    curl http://pear.php.net/go-pear | sudo php

We'll be presented with some text, once we've entered our admin password, along the lines of:

  Welcome to go-pear!
 
  Go-pear will install the 'pear' command and all the files needed by
  it.  This command is your tool for PEAR installation and maintenance.
 
  Go-pear also lets you download and install the following optional PEAR
  packages: PEAR_Frontend_Web-beta, PEAR_Frontend_Gtk2, MDB2.
 
  If you wish to abort, press Control-C now, or press Enter to continue: 
 
  HTTP proxy (http://user:password@proxy.myhost.com:port), or Enter for none::

We probably don't need a proxy, so you can just hit enter, at which point we'll be asked the following:

  Below is a suggested file layout for your new PEAR installation.  To
  change individual locations, type the number in front of the
  directory.  Type 'all' to change all of them or simply press Enter to
  accept these locations.
 
   1. Installation prefix ($prefix) : .
   2. Temporary files directory     : $prefix/temp
   3. Binaries directory            : $prefix/bin
   4. PHP code directory ($php_dir) : $prefix/PEAR
   5. Documentation base directory  : $php_dir/docs
   6. Data base directory           : $php_dir/data
   7. Tests base directory          : $php_dir/tests
 
  1-7, 'all' or Enter to continue:

We want to change the installation prefix, to make sure we're putting this stuff into /usr/local - I found that I had to do this explicitly, because the default value for $prefix, ended up with me getting build errors.

Once we've added this, we'll get a load of text flying past as PEAR is built, and end up with some text saying something like:

    WARNING!  The include_path defined in the currently used php.ini does not
    contain the PEAR PHP directory you just specified:
 
    If the specified directory is also not in the include_path used by
    your scripts, you will have problems getting any PEAR packages working.
 
    Would you like to alter php.ini ? [Y/n] :

Why yes, we would like this added. This will add this snippet to your php.ini file:

    ;***** Added by go-pear
    include_path=".:/usr/local/PEAR"
    ;*****

One important thing here - now that you have this, make sure your PEAR

Now, any thing you add to pear will be available in future, so installing PHP Unit for unit testing is as simple as calling this on the commandline (note):

  pear install phpunit/PHPUnit

In a similar fashion to Rubygems, these libraries of classes are available by calling require_once as if you were in the path /usr/local/PEAR, so to pull in PHP Unit, you just type:

  require_once 'PHPUnit/Framework.php';

Setup PECL with PHP 5.2

Now that we have PEAR setup, we should spend a it of time on making PECL simple to administer without sudo. All I needed to do here was check that everything in /usr/local/Cellar/php52/ belonged to my normal user account, by calling:

    sudo chown -R /usr/local/Cellar/php52/

This does away with the need to compile things as root, and makes installing extensions like Memcache, Mongodb, APC, as simple as:

    pecl install apc

If you find yourself having trouble here, you may want to check the permissions on any extra directories created during the PECL installation process - I had a couple of issues because some extensions had been installed as root earlier when following Hunter Ford's instructions at Cupcake with Sprinkles, which stopped my user account being able to install into the nested directory structure, because my account was trying to put files into directories owned by root.

Fetch Xdebug

If we didn't need xdebug, this would be all we needed, but sadly things aren't as simple as that. If we try calling something like pecl install xdebug, we DO get a version of debug installed, but it's not quite what we need, and doesn't seem to do anything useful. If we want this to work with MacGDBp, we need to install the version compiled by the chaps at Active state from their Remote Debugging page, that's designed to work with their own IDE, but also other tools. Once we've downloaded the tar file, we need to choose the correct version,

And then copy it to where the other extensions are:

    cp xdebug /usr/local/Cellar/php52/5.2.13/lib/php/extensions/xdebug.so

And make the relevant changes in our php.ini file for them, to point to the xdebug shared object (that's the .so suffix on extensions), and provide a few defaults as directed on the MacGDBp help page:

  ; Adding xdebug
  zend_extension=/usr/local/Cellar/php52/5.2.13/lib/php/extensions/xdebug.so
  xdebug.remote_enable=1
  xdebug.remote_autostart=1
  xdebug.remote_host=localhost
  xdebug.remote_port=9000

In short we're telling xdebug to switch on by default, and and listen on localhost:9000 for any clients that want to connect to it when we want to inspect what's happening with code.

Finally turning on MacGDBp

Still with me? Good. At long last, we can finally start looking through code in our debugger. Fire up MacGDBp, and when you run your next PHP script, you should get to see something like this pic pilfered from Particle Tree's own post on this subject:

There's far more to this, but in general the key to using the debugger is knowing how to set breakpoints, and remembering that your choices in the blue buttons are (from left to right)

• stepping into functions to see what's happening as a request is passed from function to function
• stepping out when you no longer need to see what's happening in a particular function
• ...and stepping past a particular function, without needing to look into its working at all

The green button is a fast-forward button, to take you to the next breakpoint you might have in the code, and the green power button is analogous to a power button, to refresh a connection to the debugger. You really, really should look into this post here by Tim Sabat at Particle Tree, and this one here by Matt Butcher, at TechnoSophos.

Debugging without MacGDBp

Of course, you don't need to use MacGDBp all the time, In most cases, just having a stack trace will help find the source of the error. Once you've got xdebug setup, you should get a handy track trace whenever you throw an error - if not, be sure to make sure you have the display errors setting set to 'on' in your php.ini setting (it should be around line 370 in your file, normally):

    display_errors = On

By the time you've followed these steps, you should have an easier to main installation of the LAMP stack, with a simple way to install extra PHP libraries and C extensions, a couple much more effective ways to debug than simply typing echo $variable everywhere, and an ideal environment to get on with hacking on Drupal, WordPress and any other PHP code in future.

Phew!

This guide is largely a synthesis of earlier blog posts about this subject by Justin Hileman, Matt Butcher, Tim Sabat, Hunter Ford, and Boris Gordon, I'd recommend subscribing to their blogs, as they definitely know more about PHP than I do, and they're a great source of handy info.

If there's anything that isn't clear, please let me know, so I can improve this guide - I don't want anyone else to have to blunder through working out all this themselves in future, as working all this out has taken, far, far longer than I'd like, and well, it seems churlish not to share now that I've got a setup that seems fairly stable.

Now, time to actually do something useful with it...

I put out a request today to my followers Twitter asking this question:

I've had the following services recommended to me by a number of fellow developers whose opinions I have a lot of respect for:

A couple of ex-Headshifters pointed me to Duplicity, a free tool that also looks very promising. Using it looks pretty straight forward.

    duplicity /home/my_directory scp://backup@other.host//usr/backup

A variant of it can be used to backup to Amazon S3 too, and there's a helpful blog post to show how to do this here, if you use Ubuntu or OSX.

Kalv recommended Safe, a clever Ruby gem by the Astrails crew that was originally designed to automate the backing up of Rails projects in a simple fashion.

One service that looks really interesting is though is Tarsnap, as recommended by [Jon Gilbrath][]. It's similarly simple to use, but takes care of the some of the more awkwards backup issues, and saves you having to setup your own S3 account.

  # Create an archive named "mybackup" containing /usr/home and /other/stuff:
  tarsnap -c -f mybackup /usr/home /other/stuff

The developer, Colin Percival also has written quite extensively about how it works on his own blog - he's only making a few cents per gigabyte on providing this service for people, yet it still looks to be something viable for him to run - amazing.

It looks like it's almost exactly what I'm after - the only thing missing is the option to back up storage inside the EU. This requirement is mainly one coming from data protection concerns from previous clients, because rules for processing data and storing in in the EU are different to that in the US, but given the strength of encryption, I'm not sure how much of an issue this really is, these days.

I'd really appreciate some light shed on this actually - technology is moving so much faster than law these days, it's frustrating not being able to take advantage of these kinds of services.

Anyway that's what I've found. What do you use, and why?

But no matter how many times I see it, I don't think I'll ever get used to having this snippet available to me if I get locked out of a WordPress install :

  <?php wp_set_password('new_admin_password',1); 
// this changes the password for user with an id of 1 (i.e. the admin)  ?>

Surely it shouldn't be that easy to get root on a WordPress site?

Ladies and gentlemen, this is why you should always sequester access on a database, so each app has it's own db user. If a charmless bad guy DOES gain access, they can only reck one WordPress site database, not all of them.

If you're using Vim, and you find that you need to save a file as root, the normal commands like :wq won't work. Instead you'll need to call this command:

  :w !sudo tee %

Okay that's nice. What is it actually doing?

The above command is basically saying call the write command, passing in sudo tee % as the argument. Now sudo we should be fairly comfortable with by now, leaving us just puzzling over what tee and % actually mean. Thankfully, it's not too complex.

The tee command takes the the standard input (which in this case is the contents of the file we're editing), and writes that into the filename referenced by % which is a vim shorthard for the current filename.

Again, in English please

Sadly for these purposes, my first language for thinking in still English rather than Bash or Vim, so the only way I think I'll ever retain this particular gem is to think of this in the following terms:

From within vim, take the contents of the file I'm editing, and as sudo, write it into the file I'm currently in, doing so as root.

One day this will all be second nature to me when I'm a real programmer, but til then, I'll stick with Textmate - it's massively powerful, but when you come against these strange cases, it defers to OS X's much slicker handling of permissions issues.

Why do I like it?

I think it's about as attractive as a code type tattoo is going to get

The proportions are well balanced, and there's a nice visual rhythm in there - this is largely down to the choice Bitstream Sans Mono, the typeface used for this. This typeface is a well loved font in programmer circles, who spend hours staring at monospace fonts - a lovely touch.

It's actually executable code

This particular incantation has its own story too - this cryptic combination of symbols is a working example of a forkbomb, and was presented as a piece of open source art back in 2002, by Denis Jaromil Rojo, an Italian Rastafarian developer and media activist now residing in Amsterdam. A forkbomb is a piece of self replicating code that when called, start forking itself relentlessly, until it consumes all the resources on a computer system, rendering it unusable.

For the terminally curious, here's how it works:

 
:()      # define ':' -- whenever we say ':', do this:
{        # beginning of what to do when we say ':'
    :    # load another copy of the ':' function into memory...
    |    # ...and pipe its output to...
    :    # ...another copy of ':' function, which has to be loaded into memory
         # (therefore, ':|:' simply gets two copies of ':' loaded whenever ':' is called)
    &    # disown the functions -- if the first ':' is killed,
         #     all of the functions that it has started should NOT be auto-killed
}        # end of what to do when we say ':'
;        # Having defined ':', we should now...
:        # ...call ':', initiating a chain-reaction: each ':' will start two more.

More on Forkbombs here.

RVM does some clever voodoo with symbolic links and suchlike on your box to let you switch between versions of Ruby very easily on the command line simply by typing RVM use ruby-version when you want to use a particular flavour of Ruby. So If I wanted to run MacRuby instead of the usual version of Ruby 1.8.7 that uses the MRI Interpreter, I'd simply check what versions of Ruby I had installed on my box like so :

 
chrisadams@edam[/usr/local/Library]
[7:52]:rvm list
 
   jruby-1.4.0 [ [x86_64-java] ]
   macruby-nightly [ ]
   ree-1.8.7-2010.01 [ ]
=> ruby-1.8.7-tv1_8_7_249 [ x86_64 ]
=> (default) ruby-1.8.7-tv1_8_7_249 [ x86_64 ]
   system [ x86_64 i386 ppc ]

And then say which Ruby I want to use for the rest of the session I have open:

[7:52]:rvm use macruby-nightly
 
  Now using macruby nightly

And if I wanted to switch back, I'd just type

chrisadams@edam[/usr/local/Library]
[7:54]:rvm use default        
 
Now using default ruby.

This is extremely handy, except if you're using Textmate, which by default, will happily use a version of Ruby that pays no attention to your version switching japery, making testing and development rather less fun.

Fortunately, there are now some tools to make RVM work with everyone's favourite mac only OS X editor now. Here's how to to make the two work together easily:

First of all tell RVM to setup your symlink:

    rvm 1.8.7 --symlink textmate

Now, you'll need to tell Textmate to use this version of Ruby instead by a) setting a shell variable, and then forcing Textmate to use this Ruby instead, by moving the Builder class, that normally sets up it's Ruby shell environment:

Here's where you should set your shell variable in Textmate:

Once you have that, swap out the Builder as mentioned before, then restart Textmate.

    cd /Applications/TextMate.app/Contents/SharedSupport/Support/lib/ ; mv Builder.rb Builder.rb.backup

And that should be about it. You can easily test that this worked in Textmate by opening a new file containing only the following code snippet:

    puts RUBY_DESCRIPTION

... then either saving the file with a .rb extension, or set the syntax colouring for Ruby, then hitting command +

If it worked, you should see something like this:

If not, don't despair - there's lots of useful docs on the RVM site itself, and the irc channel, #RVM is full of wonderfully helpful types.

Now go, armed with this knowledge and make Textmate and RVM to play nicely together again.

This meant I had to use the zip tool, which I can never remember how to use, hence this memory jogging post.

Now if someone's using a Mac or is on Linux, the normal command line approach to to tar up a directory to send to someone goes like this:

tar -cvzf my_tarfile.tar.gz a_directory_to_tar and_otherone_here

We're calling tar here, with the -create (c), file (f), compress (z) oh, and the verbose (f), flag. Think of that string like saying to the computer: "create the tarfile my_tarfile putting it into a file we just created, verbosely, while compressing it, from a_directory_to_tar and and_otherone_here".

As commands go, the order feels a bit weird at first, but you get the hang eventually.

Where zip is different

Using zip isn't too different really, but it you need to remember to tell it to zip recursively (why is this not default behaviour?). If we call command:

zip archive a_directory_to_tar and_otherone_here

We'll end up with an rather useless empty file called archive.zip. We need to pass in the recursive flag

zip -r archive a_directory_to_tar and_otherone_here

There's also no need to add the .zip file extension - this is added for you as a courtesy. Well, after that ridiculous non-recursive default shenanigans, frankly that's the least it could do...

Anyway, there you have it. The differences you between them if you don't want to dive into the manpages just to package up a folder for somebody, or look at this linux.about.com.