Gone are the days of source editing on a live server, uploading files to the server manually via FTP and doing post-install stuff by hand. Or at least that should be the case. For single server projects that may still be manageable (yet unbearably annoying), but if a project requires more than one server it is a real pain.

That is one of the reasons I recently have been trying to automate deployment of my projects. The deployment process can be run with one command and will happen mostly without human intervention (YMMV depending on project). If any problems occur on the live version, a patched version can be deployed much quicker than a human could - reducing deployment time from 30+ minutes down to just a few minutes.

Such tools are definitely important for System Administrators, however I would recommend Software Developers to look into it as well, as that may be beneficial for your projects as well.

Today we’ll be looking into Fabric, but there are larger packages with a slightly different purpose (e.g. Puppet or Chef are intended for server management) and they can still be used for this purpose.

Setting up

I will assume that you have your project hosted in a version control system, such as Git (Github) or Mercurial (BitBucket), that can be accessed by your target server. If you don’t use version control - you really really should (Why should I use version control?). If you don’t want to use version control, you still can use Fabric, you’ll just have to do more tailoring of deployment scripts for your situation.

We’ll be using git, but any other version control system would work just as well, just please set up deployment keys for your remote server so that it really can access the repo: For BitBucket, For Github.

Fabric is available through PIP. Run this to install Fabric:

pip install fabric

Once that’s done, run fab to verify that fabric was successfuly installed. Output should be similar to:

Fatal error: Couldn't find any fabfiles!

Remember that -f can be used to specify fabfile path, and use -h for help.

Aborting.

Then go to your project folder and create fabfile.py which will hold deployment code for our project.

Our first fabfile

Before we do that, there are few things we’ll need to know.

fab task1 task2 ... taskn will look for a fabfile.py nearby and will try to execute functions with names task1, task2, …, taskn within the fabfile. For more extensive list of usage look into fab -h.

Fabric uses SSH as transport to run commands on remote machines (which usually are UNIX commands). Think of it as automated version of yourself running commands via terminal.

Useful list of commands:

• fabric.context_managers.cd(path) - change directory, UNIX, remote
• fabric.context_managers.lcd(path) - cd analog, local
• fabric.operations.local(command, capture=False, shell=None) - execute command on local machine
• fabric.operations.run(command, shell=True, ...skipped...) - execute command on remote machine
• fabric.operations.get(remote_path, local_path=None) - download files from remote machine
• fabric.operations.pu(local_path, remote_path=None, use_sudo=False, ...skipped...) - upload files to remote machine

That should be enough to get started. For more extensive documentation visit HERE.

And here’s our first fabfile:

from __future__ import with_statement
from fabric.api import *
from fabric.contrib import files
from fabric.contrib.console import confirm

env.hosts = ['web1']
DBUSER = 'root'
DBNAME = 'prod'

def production(version=None):
    with cd('<project path>'):
        run('git pull')
        if not version:
            run('git checkout origin master')
        else:
            run('git checkout live-%s' % version) # use specific tag
        if files.exists('sql/latest') and confirm('Run latest SQL upgrade?'):
            with cd(path):
                run('mysql -u %s -p %s < install.sql' % (DBUSER, DBNAME))
        else:
            print('Skipping SQL upgrade.')

The fabfile is rather simple and its goal is to pull the latest (or a specific) version from git and if needed run SQL upgrade scripts.

env.hosts holds address of the target server. with is a python context manager, cd helps to simplify paths.

Fabric will pull requested version from git and check if there are any SQL upgrade files. If there are - it will ask you if you want to run them and do so. Even with commands we’ve covered the script can be easily extended to do more complex things - build from source, sync static resources to backup servers.

This is script is pretty much what I use for my own projects, just slightly modified (e.g. I pull the version from source code, do automatic minification), but so far it is sufficiently convenient.

Last thoughts

I’ve only started using Fabric recently and wouldn’t call myself an experienced user, but so far it’s been a great experience. I like the level of automation that can achieved with it.

One problem I’ve experienced and haven’t been able to find a solution is with SSH key management. For some reason it will use a wrong key (ignoring SSH config) and won’t change to a different one. If you know a solution for this - I would love to hear it.

For official documentation go to Fabric homepage. They have a good tutorial, which you may find more understandable compared to my quick overview of my usage.

For my next adventure I will probably look into Puppet. Justin Weissig on Sysadmincasts.org did an episode on Learning Puppet with Vagrant - it’s very interesting.

If you have any questions or remarks about the post - please let me know either by email or in the comments below. I would love to hear from you.