by June 29, 2007

How to install memcache on Debian Etch

Memcache is a technology which caches objects in memory where your web application can get to them really fast. It is used by sites such as Digg.com, Facebook.com and NowPublic.com and is widely recognized as an essential ingredient in scaling any LAMP application to handle enormous traffic. Drupal has modules to leverage memcache, but setting up the right tools on a Linux server can be daunting. This article leads you through that process, step by step.This article walks through the steps needed to install the libevent, memcached and PECL memcache libraries on Debian 4.0 (Etch). These are the prerequisites to being able to use the Drupal memcache module.

Memcache consists of two parts; a server library which runs the caching daemon and a client library which allows PHP to interact with the server. The server library is called memcached and it depends on the libevent library, so the first step in the installation process is getting a recent copy of libevent.

Getting and installing libevent

Use ssh to log onto your server and su or sudo to root:

# sudo bash

I use the locate command a lot to keep track of where things are, and to make sure its database is up to date I sometimes need to run updatedb

# updatedb

Now I can use locate to find out what libevent files are already present on my Debian installation. This is what it looks like if libevent has been installed using the apt-get package manager.

# locate libevent /usr/share/doc/libevent1 /usr/share/doc/libevent1/changelog.Debian.gz /usr/share/doc/libevent1/copyright /usr/lib/libevent-1.1a.so.1.0.2 /usr/lib/libevent-1.0d.so.1 /usr/lib/libevent-1.0e.so.1 /usr/lib/libevent.so.1 /usr/lib/libevent-1.1.so.1 /usr/lib/libevent-1.1a.so.1 /var/cache/apt/archives/libevent1_1.1a-1_i386.deb /var/lib/dpkg/info/libevent1.shlibs /var/lib/dpkg/info/libevent1.list /var/lib/dpkg/info/libevent1.postinst /var/lib/dpkg/info/libevent1.postrm /var/lib/dpkg/info/libevent1.md5sums

If you get similar results from running the locate command, you need to follow the next instructions on removing this libevent version. If locate doesn't find any files, you don't have libevent on your system and you can safely skip the next section.

Removing an old libevent

Removing libevent is easy.

# apt-get remove --purge libevent1

Note: That's the number one on the end... it looks a lot like the letter L.

Now check to make sure everything is gone.

# locate libevent /var/cache/apt/archives/libevent1_1.1a-1_i386.deb

Acquiring and compiling the libevent source

From the libevent website find the URL to the tarball (tgz file) of the latest libevent release, which at the time of this writing is 1.3b. Copy the URL. Then move to /usr/local/src use wget to fetch the tarball onto your server.

Unpack the tarball:

# tar zxvf libevent-1.3b.tar.gz

Configure, compile and install libevent:

# cd libevent-1.3b # ./configure # make && make install

The path information for the new libevent libraries has to be added to the ld configuration. To do this you need to create a new file /etc/ld.so.conf.d/libevent-i386.conf which contains the text /usr/local/lib/

# vi /etc/ld.so.conf.d/libevent-i386.conf ## in vi type "i" to go into insert mode /usr/local/lib/ ## then type esc :wq

To make the change to ld get loaded, use the ldconfig command:

# ldconfig

Getting and installing memcached

The current release of the memcached server daemon is 1.2.2. The Debian memcached package offers only 1.1.12-1 which is inadequate for Drupal's use. Fortunately the steps for acquiring, compiling and installing the memcached library are shorter than those for libevent.

# cd /usr/local/src # wget http://danga.com/memcached/dist/memcached-1.2.2.tar.gz # tar zxvf memcached-1.2.2.tar.gz # cd memcached-1.2.2 # ./configure # make && make install

At this point you should have a working memcached daemon library. To test this, try starting one.

# memcached -u www-data -vv

The -u www-data flag tells it to run as the same user that your webserver runs. I'd like reader feedback on whether this is a safe user to run under, and if not, what a better alternative is. The -vv flag tells it to be very verbose and log virtually everything that happens to the stdout.

# memcached -u www-data -vv slab class 1: chunk size 80 perslab 13107 slab class 2: chunk size 100 perslab 10485 ... snip ... slab class 38: chunk size 323000 perslab 3 slab class 39: chunk size 403752 perslab 2 slab class 40: chunk size 504692 perslab 2 <3 server listening

Testing the memcached daemon with telnet

If you see output like the above you can be pretty sure that memcached is working for you. If you want to get a better feel for what it is doing you can follow these next steps which show you how to add and fetch data from the cache using the telnet program. This isn't part of the installation, so feel free to skip this section.

The memcached daemon runs on port 11211 by default, and there is no authentication or other protection which separates memcached from the outside world. Thus you should at this moment be able to open a telnet connection with your daemon from any machine on the Internet (which of course means that you need a firewall to protect you from such access by malicious attackers. See this howtoforge.com article on configuring iptables). If your machine is running on the IP 1.2.3.4, you can open another shell, from the same machine the memcached daemon is running on or another one, with the following command:

# telnet 1.2.3.4 11211

You can now use memcached commands to set data. The command looks like this:

set <key> <flag> <exptime> <bytes>\r\n

The flag is an arbitrary number that you can use in your client logic. It is intended to be metadata that you can assign to each cached object. In the examples this is shown as 1 but here it has no meaning. I set the exptime to zero (never expire), and the bytes to the number of characters I want to store. Note in the last example that if the bytes and the number of characters don't match, an error occurs.


set test2 1 0 2
ab
STORED
set test3 1 0 3
abc
STORED
get test2
VALUE test2 1 2
ab
END
get test3
VALUE test3 1 3
abc
END
set test4 1 0 2 
abcde
CLIENT_ERROR bad data chunk
ERROR
  

If you run these commands you should see them being executed in the other shell that still has the memcached daemon running. You are replicating the role of the PECL memcache extension which we will install next.

Getting and installing PECL memcache

Before getting and installing the PECL memcache library, you should check to see if you have the PHP development tools installed which are needed to compile PHP extensions. Back in the shell running the memcached daemon, press Ctrl+c to abort the daemon's execution. Then type:

# which phpize /usr/bin/phpize

If you don't get a value for which phpize, then you need to install the php5-dev package.

# apt-get install php5-dev

Now navigate back to the /usr/local/src directory, grab the PECL memcache tarball and unpack it.

# cd /usr/local/src # wget http://pecl.php.net/get/memcache-2.1.2.tgz # tar zxvf memcache-2.1.2.tgz

Before we configure, compile and install it, we need to address a bug in the configure script and help it find our php includes by making a symlink from the place where it is going to look for them to the place where they really are:

# ln -s /usr/include/php5 /usr/include/php

Now we can get on with our business:

# cd memcache-2.1.2 # phpize # ./configure # make && make install

Now we have to make sure PHP loads the newly built memcache.so library by adding the following line to php.ini:

extension=memcache.so

You can achieve this either by editing the file directly or by executing the following command:

# echo "extension=memcache.so" >> /etc/php5/apache2/php.ini

Now restart Apache:

# /etc/init.d/apache2 restart

Running phpinfo() on your webserver should now confirm that memcache is installed:

The output of phpinfo() showing that memcache is successfully installed

Starting memcached when the server boots

You want to design your system so that everything involved in hosting a website boots on its own when your server machine starts up. On Debian this is achieved by adding scripts to the /etc/init.d folder (such as the one we just ran to restart Apache). Create a file /etc/init.d/memcached to start whatever memcached daemons you wish to use. The optimal number of daemons depends on your needs, but generally you want one for each cache table in your Drupal database because this makes clearing the cache on any of those tables less disruptive to the rest of your cache. If you are running a fairly straightforward site that uses CCK and Views, this automatically means six memcached daemons for optimal performance. If that seems like a lot to manage, you can use fewer, even one, and still get the bulk of memcache's performance increase. Here's what your /etc/init.d/memcached script file might look like with six daemons:


#!/bin/sh -e

memcached -u www-data -p 11211 -m 2 -d
memcached -u www-data -p 11212 -m 2 -d
memcached -u www-data -p 11213 -m 2 -d
memcached -u www-data -p 11214 -m 2 -d
memcached -u www-data -p 11215 -m 2 -d
memcached -u www-data -p 11216 -m 2 -d
  

To keep it simple this script assigns 2M memory to each daemon. You'll want to monitor these daemons using the Drupal memcache_admin module to see whether or not they are full, and if they are, increase the size allocations accordingly.

Make sure that the script can be executed:

# chmod u+x /etc/init.d/memcached

And then try running it and use ps to see if it worked:

# /etc/init.d/memcached # ps -A | grep memcached 23846 ? 00:00:00 memcached 23848 ? 00:00:00 memcached 23850 ? 00:00:00 memcached 23852 ? 00:00:00 memcached 23854 ? 00:00:00 memcached 23856 ? 00:00:00 memcached

If you ever need to stop your memcached daemons, use the killall command:

# killall memcached # ps -A | grep memcached

That's it! Don't forget to configure your firewall so that only the good guys can access your memcached instance. The next steps are to check out the Drupal advcache and memcache modules which leverage the tools discussed in this article to make your Drupal site blazing fast.

Sign up for the Lullabot Newsletter and get our latest articles and more direct to your inbox

We can partner with you!

We’ve accomplished some amazing things for some really great clients. We’d love to talk about how we can help your next project!

Tell us about your project

You can be a ’bot!

We have an amazing team of friendly experts in a number of disciplines. We work hard, have lots of fun together, and take great care of our team.

See our open positions