Lullabot Ideas

We know stuff. We empower you to know stuff too.

Tag it Up!

Article by Jerad BitnerJuly 26, 2010 - 9:21am

Release management with version control
Managing the code on your servers is a very important part of the release process. Over the years we've tried many different ways and have found the below process as a tried and true starting point, but here we'll focus on revision control (Subversion or Git) and the use of tags in each for any given project. Since many of our clients are large traffic websites and need multiple web heads, we also have some standard practices and scripts that we like to use on our projects to help keep the web heads synchronized. So let's take a look.

Development Branches

We have a project going right now that has two separate teams working on the code base. We are primarily concerned with the Drupal side of the project, while the other team is concerned with the CiviCRM portion of the project. This is a great example because each of us have our own respective development branches that we are working within, and which we each are responsible for keeping up to date with the Trunk of the project.
Branches
- civicrm
- drupal
- trunk
The CiviCRM branch houses a full upgrade of CiviCRM to the latest version and any related code to make this happen. The Drupal branch has new feature requests, a brand new theme, and some UX improvements as well as a whole lot of restructuring that is now housed in code as well as an upgrade path to migrate Domain Access to Private module. Trunk is always the latest version of code that is running on the live site. Each time a fix is made to Trunk which is to be pushed to the live site, a Tag is created. The live site is run from the latest Tag at all times so that we have an easily defined point to roll back to in case there is a problem with the update.

Release Tags

As I mentioned, the live site is always running from a particular Tag. Our naming structure is 'release-major.minor' (ex: release-1.2). The major version, in this case '1', refers to when we started this practice. When the major improvements from one of the development branches makes it's way into Trunk, the major version of the release will be incremented (ex: release-2.0).

Keeping the Web Heads Synchronized

The site is running on four web servers which are served through a load balancer which determines what web server to use for a particular request. Each of the web server's must be mirrored exactly when the code base is updated so that they're always serving the same code. To do this we first have a 'staging' area where we can do our Subversion work. This staging area is where we have a checkout of the code with all relevant settings and symlinks in which we can switch to the next Tag. The process looks something like this:
$ cd /usr/drupal_prod/public_html
$ svn switch [repository-location]/tags/release-2.0/public_html
$ sync-web-heads
The sync-web-heads command is a reference to a script we have that basically takes the code within /usr/drupal_prod/public_html and rsyncs the contents to each server.
Note: the following is a partial script, but should get the idea across.
# Define the source to be copied from
SOURCE="/usr/drupal_prod/public_html"

# Define the destination directory on each server
DEST="/var/www/your-website/public_html"

# List your IP Addresses separated by a space
CLUSTER=""

# Generate rsync command
COMMAND="rsync -av --delete --exclude=.ssh --exclude=CVS --exclude=.svn $SOURCE $DEST"

# Check if each cluster member is alive
HOSTS=""
for HOST in $CLUSTER; do
  ping -t 1 -c 1 -s 1 $HOST > /dev/null 2>&1
  if [ $? -ne 0 ]; then
    echo
    echo "*************************************************"
    echo "Host: $HOST either does not exist or is down."
    echo "Process resumes in 5 secs or Control-C to cancel"
    echo "*************************************************"
    sleep 5
  else
    HOSTS="$HOSTS $HOST"
  fi
done

# Execute command
for HOST in $HOSTS; do
  echo
  echo " --- $HOST --- "
  ssh $HOST "$COMMAND"
  echo " --- done --- "
done
wait

Comments

Jamie (not verified) on July 26, 2010 - 3:37pm

Nice Article!

I use a very similar procedure for maintaining clients sites and it works out great. One question though. Any reason why you are doing the excludes on RSYNC for .svn and CVS, instead of just using the -C (-cvs-exclude) flag?

July 26, 2010 - 5:24pm Jerad Bitner

no particular reason

I think we just didn't see that option and already knew about --exclude

pat (not verified) on July 26, 2010 - 6:30pm

settings files?

Great article! Love this approach.

Do you keep your settings file(s) (or any files containing sensitive if such as passwords) in svn, or are they already on live site(s) and they don't get overwritten?

July 26, 2010 - 6:48pm Jerad Bitner

ooc

Out of control! Configurations, like $conf['whatever'] are kept within version control, with a settings.inc which is included by the settings.php, but keeping database usernames and passwords out of source control are always a good practice.

dalin (not verified) on July 27, 2010 - 1:01am

Excluding the CVS folders is

Excluding the CVS folders is probably not necessary and in fact troublesome in that cvs_deploy module won't be able to figure out what version your modules are running.

Ann (not verified) on August 13, 2010 - 2:30pm

Useful?

Very interesting stuff, if this is a solutions for well known organization problems, we'll maybe try it :-)

A big wish would be something like a mind map for orga parts.

But anyways: thanks a lot so far!

Anonymous (not verified) on August 23, 2010 - 9:17pm

To utilize a source control

To utilize a source control like Versions or Git do you need command line access to the live web site server?