by Robert Douglass on June 29, 2007 // Short URL

How to install memcache on Debian Etch

Memcache is a technology which caches objects in memory where your web application can get to them really fast. It is used by sites such as Digg.com, Facebook.com and NowPublic.com and is widely recognized as an essential ingredient in scaling any LAMP application to handle enormous traffic. Drupal has modules to leverage memcache, but setting up the right tools on a Linux server can be daunting. This article leads you through that process, step by step.This article walks through the steps needed to install the libevent, memcached and PECL memcache libraries on Debian 4.0 (Etch). These are the prerequisites to being able to use the Drupal memcache module.

Memcache consists of two parts; a server library which runs the caching daemon and a client library which allows PHP to interact with the server. The server library is called memcached and it depends on the libevent library, so the first step in the installation process is getting a recent copy of libevent.

Getting and installing libevent

Use ssh to log onto your server and su or sudo to root:

# sudo bash

I use the locate command a lot to keep track of where things are, and to make sure its database is up to date I sometimes need to run updatedb

# updatedb

Now I can use locate to find out what libevent files are already present on my Debian installation. This is what it looks like if libevent has been installed using the apt-get package manager.

# locate libevent
/usr/share/doc/libevent1
/usr/share/doc/libevent1/changelog.Debian.gz
/usr/share/doc/libevent1/copyright
/usr/lib/libevent-1.1a.so.1.0.2
/usr/lib/libevent-1.0d.so.1
/usr/lib/libevent-1.0e.so.1
/usr/lib/libevent.so.1
/usr/lib/libevent-1.1.so.1
/usr/lib/libevent-1.1a.so.1
/var/cache/apt/archives/libevent1_1.1a-1_i386.deb
/var/lib/dpkg/info/libevent1.shlibs
/var/lib/dpkg/info/libevent1.list
/var/lib/dpkg/info/libevent1.postinst
/var/lib/dpkg/info/libevent1.postrm
/var/lib/dpkg/info/libevent1.md5sums

If you get similar results from running the locate command, you need to follow the next instructions on removing this libevent version. If locate doesn't find any files, you don't have libevent on your system and you can safely skip the next section.

Removing an old libevent

Removing libevent is easy.

# apt-get remove --purge libevent1

Note: That's the number one on the end... it looks a lot like the letter L.

Now check to make sure everything is gone.

# locate libevent
/var/cache/apt/archives/libevent1_1.1a-1_i386.deb

Acquiring and compiling the libevent source

From the libevent website find the URL to the tarball (tgz file) of the latest libevent release, which at the time of this writing is 1.3b. Copy the URL. Then move to /usr/local/src use wget to fetch the tarball onto your server.

# cd /usr/local/src
# wget http://monkey.org/~provos/libevent-1.3b.tar.gz

Unpack the tarball:

# tar zxvf libevent-1.3b.tar.gz

Configure, compile and install libevent:

# cd libevent-1.3b
# ./configure
# make && make install

The path information for the new libevent libraries has to be added to the ld configuration. To do this you need to create a new file /etc/ld.so.conf.d/libevent-i386.conf which contains the text /usr/local/lib/

# vi /etc/ld.so.conf.d/libevent-i386.conf
## in vi type "i" to go into insert mode
/usr/local/lib/
## then type esc :wq

To make the change to ld get loaded, use the ldconfig command:

# ldconfig

Getting and installing memcached

The current release of the memcached server daemon is 1.2.2. The Debian memcached package offers only 1.1.12-1 which is inadequate for Drupal's use. Fortunately the steps for acquiring, compiling and installing the memcached library are shorter than those for libevent.

# cd /usr/local/src
# wget http://danga.com/memcached/dist/memcached-1.2.2.tar.gz
# tar zxvf memcached-1.2.2.tar.gz
# cd memcached-1.2.2
# ./configure
# make && make install

At this point you should have a working memcached daemon library. To test this, try starting one.

# memcached -u www-data -vv

The -u www-data flag tells it to run as the same user that your webserver runs. I'd like reader feedback on whether this is a safe user to run under, and if not, what a better alternative is. The -vv flag tells it to be very verbose and log virtually everything that happens to the stdout.

# memcached -u www-data -vv
slab class   1: chunk size     80 perslab 13107
slab class   2: chunk size    100 perslab 10485
... snip ...
slab class  38: chunk size 323000 perslab     3
slab class  39: chunk size 403752 perslab     2
slab class  40: chunk size 504692 perslab     2
<3 server listening

Testing the memcached daemon with telnet

If you see output like the above you can be pretty sure that memcached is working for you. If you want to get a better feel for what it is doing you can follow these next steps which show you how to add and fetch data from the cache using the telnet program. This isn't part of the installation, so feel free to skip this section.

The memcached daemon runs on port 11211 by default, and there is no authentication or other protection which separates memcached from the outside world. Thus you should at this moment be able to open a telnet connection with your daemon from any machine on the Internet (which of course means that you need a firewall to protect you from such access by malicious attackers. See this howtoforge.com article on configuring iptables). If your machine is running on the IP 1.2.3.4, you can open another shell, from the same machine the memcached daemon is running on or another one, with the following command:

# telnet 1.2.3.4 11211

You can now use memcached commands to set data. The command looks like this:

set <key> <flag> <exptime> <bytes>\r\n

The flag is an arbitrary number that you can use in your client logic. It is intended to be metadata that you can assign to each cached object. In the examples this is shown as 1 but here it has no meaning. I set the exptime to zero (never expire), and the bytes to the number of characters I want to store. Note in the last example that if the bytes and the number of characters don't match, an error occurs.

set test2 1 0 2
ab
STORED
set test3 1 0 3
abc
STORED
get test2
VALUE test2 1 2
ab
END
get test3
VALUE test3 1 3
abc
END
set test4 1 0 2
abcde
CLIENT_ERROR bad data chunk
ERROR

If you run these commands you should see them being executed in the other shell that still has the memcached daemon running. You are replicating the role of the PECL memcache extension which we will install next.

Getting and installing PECL memcache

Before getting and installing the PECL memcache library, you should check to see if you have the PHP development tools installed which are needed to compile PHP extensions. Back in the shell running the memcached daemon, press Ctrl+c to abort the daemon's execution. Then type:

# which phpize
/usr/bin/phpize

If you don't get a value for which phpize, then you need to install the php5-dev package.

# apt-get install php5-dev

Now navigate back to the /usr/local/src directory, grab the PECL memcache tarball and unpack it.

# cd /usr/local/src
# wget http://pecl.php.net/get/memcache-2.1.2.tgz
# tar zxvf memcache-2.1.2.tgz

Before we configure, compile and install it, we need to address a bug in the configure script and help it find our php includes by making a symlink from the place where it is going to look for them to the place where they really are:

# ln -s /usr/include/php5 /usr/include/php

Now we can get on with our business:

# cd memcache-2.1.2
# phpize
# ./configure
# make && make install

Now we have to make sure PHP loads the newly built memcache.so library by adding the following line to php.ini:

extension=memcache.so

You can achieve this either by editing the file directly or by executing the following command:

# echo "extension=memcache.so" >> /etc/php5/apache2/php.ini

Now restart Apache:

# /etc/init.d/apache2 restart

Running phpinfo() on your webserver should now confirm that memcache is installed:

The output of phpinfo() showing that memcache is successfully installed

Starting memcached when the server boots

You want to design your system so that everything involved in hosting a website boots on its own when your server machine starts up. On Debian this is achieved by adding scripts to the /etc/init.d folder (such as the one we just ran to restart Apache). Create a file /etc/init.d/memcached to start whatever memcached daemons you wish to use. The optimal number of daemons depends on your needs, but generally you want one for each cache table in your Drupal database because this makes clearing the cache on any of those tables less disruptive to the rest of your cache. If you are running a fairly straightforward site that uses CCK and Views, this automatically means six memcached daemons for optimal performance. If that seems like a lot to manage, you can use fewer, even one, and still get the bulk of memcache's performance increase. Here's what your /etc/init.d/memcached script file might look like with six daemons:

#!/bin/sh -e

memcached -u www-data -p 11211 -m 2 -d
memcached -u www-data -p 11212 -m 2 -d
memcached -u www-data -p 11213 -m 2 -d
memcached -u www-data -p 11214 -m 2 -d
memcached -u www-data -p 11215 -m 2 -d
memcached -u www-data -p 11216 -m 2 -d

To keep it simple this script assigns 2M memory to each daemon. You'll want to monitor these daemons using the Drupal memcache_admin module to see whether or not they are full, and if they are, increase the size allocations accordingly.

Make sure that the script can be executed:

# chmod u+x /etc/init.d/memcached

And then try running it and use ps to see if it worked:

# /etc/init.d/memcached
# ps -A | grep memcached
23846 ?        00:00:00 memcached
23848 ?        00:00:00 memcached
23850 ?        00:00:00 memcached
23852 ?        00:00:00 memcached
23854 ?        00:00:00 memcached
23856 ?        00:00:00 memcached

If you ever need to stop your memcached daemons, use the killall command:

# killall memcached
# ps -A | grep memcached

That's it! Don't forget to configure your firewall so that only the good guys can access your memcached instance. The next steps are to check out the Drupal advcache and memcache modules which leverage the tools discussed in this article to make your Drupal site blazing fast.

Robert Douglass

Comments

robert

If you prefer to use the

If you prefer to use the unstable there is no problem with that. Compiling your own is one more level of control (and one more bit of work), so each has to decide which is best. The stable versions that you link to, though, are really outdated and not recommended.

Reply

Khalid -- 2bits

Ubuntu server has pretty current packages

One reason I like Ubuntu, even on servers is that they offer more current packages than Debian stable, yet you get all the benefit of upgrading via apt.

Here are the versions that come with Feisty (7.04) on amd64:

  • memcached 1.1.12-1
  • php5-memcache 2.0.1-1ubuntu1

Compiling from source has its benefits, but is labor intensive to keep current.

Reply

Gwen

fedora

Fedora has the 1.2.3 memcached and pecl packages. You can install it as follows:

$ su -
$ yum install memcached php-devel php-pear
$ pear install -f pecl/memcache
Reply

dman

Wow, it seems to have worked

Man I love keylogs like this. it Really really helped step me through what looks like a really daunting process.

I did encounter a hiccup in the middle there with the php_session problem.
Even with that fix in, it refused to believe that the session library was available.
Eventually I had to flag it inside the configure script itself by saying

PHP_SESSION_SHARED=yes

around the bottom of the
if test "$PHP_MEMCACHE_SESSION"
block.

Your explanation made me feel that it was safe enough to just over-ride the faulty library check - after I'd ensured it was indeed found where it should be.

WOW. Now to see if I can make it work on Drupal

Reply

dman

PS

I'd also like to add that my box (ubuntu feisty) didn't have a /etc/ld.so.conf.d/ directory to put libevent-i386.conf into (and I wasn't able to fully understand what ld.so was anyway) but I think I got the same effect by creating a file ld.so.conf which does the job.

echo /usr/local/lib/ >> /etc/ld.so.conf

I may be totally wrong, but I got the tests to run OK.
Now for benchmarking. Reply

Anonymous

error when trying to test memcached

Here's the error I get when I run:

memcached -u www-data -vv

memcached: error while loading shared libraries: libevent-1.3b.so.1: cannot open shared object file: No such file or directory

Any ideas? I'm trying this on CentOS4

Reply

robert

When I ran into this on

When I ran into this on Debian it meant that memcached's configure file wasn't respecting the --with-libevent flag (I think that's the one... speaking from memory here) and was looking for it in /bin and /lib instead of /usr/local/bin /usr/local/lib where it had been installed after I compiled it. Try compiling libevent to /usr and /lib (you can set a path during ./configure ... to read about your configure options you can run ./configure --help in the libevent source directory). In any case, your problem stems from the fact that memcached is looking in the wrong place for libevent.

Reply

palik

whoa!

gr8 tutorial, and just when i bought debian vps server - that is what i call "just in time"

Reply

Anonymous

The Cient won't work for me

The Cient won't work for me if I compile php with the --enable-versioning option, if anybody runs into this, just skip it compiling the php sources...

Reply

Chris

Thank you!

I've spent the last 14 hours trying to figure out why the client wouldn't work for me on my custom compiled php on Ubuntu Edgy. I removed the --enable-versioning options and bingo!!

Reply

Anonymous

security notes

On OpenBSD they recommend each new server to be run as a separate account. They create users as _postgresql and _memcached. I guess this is good practice. If one service is compromised it's harder to affect the other.

If you run memcached on the same machine as the web-server you can also bind the server to the loopback interface (127.0.0.1).

Reply

John Handelaar

Breaking apt for fun and profit?

A smarter approach would be to get libevent1 as a Debian source package from the 'testing' repository, and compile it into a binary .deb under etch. "Backporting" keeps apt entirely in tact -- under Debian distros, /usr belongs to apt, and you shouldn't be spraying binaries around in there.

If something you install later wants libevent1 as a dependency and you've told apt you've just purged it? That stuff you just suggested will get reinstalled and downgraded without so much as a polite warning.

Backporting from source works for memcached, too.

Reply

sholdowa

Comments from a sysadmin...

If replacing libraries, then don't leave the system without them while you build the new ones. Do your configure; make, modify your ldconfig setup and only then then apt-get remove ... ; make install; ldconfig. Less chance of other stuff falling over.

Reply

Alan

Stupid question for separate db server

If you have a load balancing scenario with a couple drupal servers fronting the mysql server, does memcache go on the mysql server or each of the drupal servers?

Reply

jayjaytheluffy

memcache doesn't appear in phpinfo

I've followed carefully the above steps...yet when I ran my phpinfo, memcache doesn't appear like the one shown above.

What could be the problem?

I'm using Linux server with drupal as CMS of course, is it necessary to follow this steps?
Or am I suppose to use the INSTALLATION.txt that comes with the memcache module I downloaded in drupal.org?

Please I need an urgent reply, I've been doing it twice already and haven't succeeded, I think..

thanks

Reply

Anonymous

to jayjaytheluffy

I ran into the same mess: drupal, no memcache section displayed in phpinfo().

Then I created a separate file with phpinfo() and ran it. There I see the memcache and far more other inforation.

So the phpinfo in drupal deval module is quite limit.

Try it yourself.

Reply

Langdon

nice

may the internet gods bless your little blog. this entry was extremely helpful.

Reply

Carlos

THANK YOU!!

Hi Robert, thanks a million for this tutorial page!!! I'm a complete newbie in the Linux world (basically knew only commands cd and ls, lol) and even so I could set a memcached up and running on a Wiki server with your help. It only took some extra time for me because my installation didn't have gcc yet and I had to "yum" it first. By the way, thanks also to Gwen who posted a comment about yum, I had tried with apt-get and it didn't work for me.

Reply

Carlos

My steps

Well, I'll put here the packages I had to install, maybe it can be of help to someone. First I downloaded the latest stable versions of libevent, memcached and memcache:

cd /usr/local/src
wget http://monkey.org/~provos/libevent-1.4.8-stable.tar.gz
wget http://danga.com/memcached/dist/memcached-1.2.6.tar.gz
wget http://pecl.php.net/get/memcache-2.2.4.tgz

When I uncompressed the files to their directories and executed the ./configure's, they couldn't find the following packages, so I installed them:

yum install gcc
yum install php-devel
yum install zlib-devel
yum install re2c-devel

(When yum offers to download and install the newest versions, reply "y")

Finally, the command "memcached -u www-data -vv" didn't work for me, I had to change user to "-u root" (my login user).

Reply

mosh

gentoo

I have problem with install memcache on my server
i cant find memcache.so ....
whre is it ?

need help, mosh

Reply

gkirok

great

thank you.
adding to memories

p.s.: i think "a bug in the configure script" resolved, i didn't need to make ln

Reply

Anonymous

How to modify /etc/init.d/memcached

/etc/init.d/memcached already exists (I installed memcached via apt-get). I'm not sure how to modify it so that it opens the 6 recommend daemons, because the script looks quite involved.

Reply

Fred Wolf

That is a great tutorial

One of the clearest I read, and it works !!
Just one thing on the last step of the pecl extension install. I couldn't see the memcache section in the php.ini beacause the reference at the memcache.so library was written at the end of the ini file AFTER the memcache directives. So I put it BEFORE them and that's it !

One more thing, the article was written on the 27,june 2007 so, maybe you could need newer versions to install. I updated the script lines without any problems.

Thank you !

Fred

Reply

alexis bellido

You really need to do it this way

Thanks Robert!

I was having very strange errors on one Drupal 5 site running on an Ubuntu 9.04 server with memcache. I was getting blank pages and I noticed my logs were full of watchdog messages from memcache: 'Failed to set key ****'.

After removing the Ubuntu packages for memcache and php5-memcache I still get some 'Failed to set key' errors but no more blank pages and the site is working.

Cheers.

Reply