Home

Lullabot

Lullabot Ideas

We know stuff. We empower you to know stuff too.

Drupal Module Development Deep Dive Week

London, UK
September 20-24, 2010

Installing Varnish on Mac OS X (with MAMP or any other server)

Article by Nate HaugMay 3, 2010 - 9:29pm

Varnish is an excellent high-performance, HTTP accelerator. The technical term for Varnish is a "reverse proxy cache", meaning that it handles the requests when you visit a website acting as a cached layer of content on top of Apache. This means that after a page has been requested once from the web server, Varnish keeps a copy of that file in an ultra-fast storage so that the next time that page is requested, it returns it immediately instead of starting up Apache, PHP, MySQL, and any other technologies your web site may be built upon. If Varnish doesn't have a copy of the file or page being requested, it will request the page from the normal web server.

This article explains step-by-step instructions on how to get started with Varnish on a local Mac OS X sandbox. I personally set it up with the MAMP package, but because it doesn't make any difference what web server you use, you can use these instructions to set up Varnish in front of the built-in Mac OS X Apache or anything else you may have installed from MacPorts or compiled from source.

If you're not familiar with setting up a web server on Mac OS X, you'll need to get that working first. I'd recommend the Lullabot videocast on Installing a local web server on Mac OS X.

In this tutorial, we'll be installing Varnish from source. I haven't seen any Mac OS X binaries for Varnish out there yet, but even if they eventually become available this guide could still be useful for installing the cutting edge versions of Varnish.

Preparing for installation

The first thing you'll need to compile anything from source is a compiler, the most common one being GCC (Gnu C Compiler). This is included with a Mac OS X developer tools (better known as XCode). You'll also need MacPorts installed to give you access to the "port" command. Both of these packages are standard .dmg double-click installers.

  • Download XCode (free, but registration required)
  • Download MacPorts

Varnish requires a few libraries that aren't included with Mac OS X or are out-dated, so we need to update them first.

Run the following two commands to install automake and libtool.

$ sudo port install automake
$ sudo port install libtool

Now it's likely that Mac Ports will install these files in a directory that is not checked for executables by default. So you may need to adjust your $PATH in your shell configuration.

If you use bash as your shell (the Mac OS X default) then your shell configuration file will be named .profile or .bashrc in your Home directory. Open this file with your preferred text editor, or create an empty text file if it doesn't exist at all.

$ vi ~/.profile

Add these lines to this file. Note that you may already have a similar line in this file, just add /opt/local/bin: to the beginning of it if it's not there already.

# Add MacPorts executable directory to $PATH. First declarations take precedence over later ones.
export PATH=/opt/local/bin:$PATH

This should make it so that MacPorts executables are used over the default ones provided by Mac OS X. To confirm you can run "which automake" (the port we installed above) and see that it lives in /opt/local/bin.

$ which automake
/opt/local/bin/automake

Installation of Varnish itself

Okay, we're all now in place and ready to actually install Varnish! This part is actually quite straight-forward. The official installation instructions may also be a useful reference.

  1. Download Varnish from SourceForge.
  2. Expand the downloaded archive and switch to that download directory.
    $ cd ~/Downloads/varnish-2.0.6
  3. Now we need to compile the downloaded source code. There will be quite a bit of text following each command. As long as you don't get any blaring ERROR: warnings then the compilation was successful.
    $ ./autogen.sh
    $ ./configure
    make
    sudo make install
  4. After this, Varnish should be available. However this process places the main Varnish executable, varnishd in an unusual location. Try running the following command:
    $ which varnishd
    # What you'd expect:
    /usr/local/bin/varnishd
    # What you probably get instead:
    varnishd not found

    If varnish is found, continue on to the next step. However for most people Varnish will be installed in an unusual location. For me, this location was /usr/local/sbin/varnishd, where varnishd was the only file in that entire directory. All the other Varnish executables end up in /usr/local/bin like they should. So my preference is to move varnishd in with the rest of the files.

    sudo mv /usr/local/sbin/varnishd /usr/local/bin/varnishd

    Now varnishd should be easily found with a which command.

    $ which varnishd
    /usr/local/bin/varnishd

  5. You can now start up Varnish by using the varnishd command. This command takes a few parameters which can be useful for setting up a few basic options.

    $ sudo varnishd -a 127.0.0.1:8080 -b 127.0.0.1:80 -s file,/tmp,500M

    The -a flag is where Varnish will be accessible. This means that if I visit http://localhost:8080/ I will be using Varnish. The -b flag is where Varnish will look if it doesn't have a page in cache. Point this to your local Apache port. I have MAMP to use port 80, though the default port for MAMP is 8888 when you first install it. Lastly the -s flag is for "storage", indicating how much space Varnish should use for its own purposes and where it should keep the cache file. By default Varnish will attempt to use a whopping 50% of free space on your disk. So if you have 120GB available, it will make a 60GB swap file! Since local environments rarely require such space, the above command will use 500 MB.

  6. Open up your web brower and point it at http://localhost:8080. You should see your localhost files exactly as though you were looking directly at http://localhost. However, inspecting the headers you should see that the pages are being served by Varnish. Using FireBug for Firefox, you can confirm this by checking under the "Net" tab. You'll be looking for a header that says Via: 1.1 varnish.
    varnish-headers.png
    The Varnish headers as display through the FireBug "Net" tab.

Post Installation

After getting Varnish installed and running the first time, you'll probably start to wonder, "okay, now what do I do with it?" Installing Varnish locally is a way of making your local sites serve up really, really fast, but it's not a good idea to use for anything other than testing your live server configurations.

Regarding how to set up your Drupal site with Varnish, we're planning more detailed Lullabot.com articles, but in the mean time these references may help you get started.

  • OSU Open Source Lab and a follow-up on the Varnish module
  • Drupal.org Varnish module

Comments

Derek "dww" Wright (not verified) on May 4, 2010 - 1:55am

sbin exists for a reason

sbin is for system binaries. varnishd isn't a command you're supposed to run from the shell -- it's a daemon (that's what the "d" at the end of the name is for). It's supposed to be started automatically for you and stay running. Just like httpd and sshd aren't normally in your PATH, neither should varnishd. Generally, a system daemon like this should be started at boot time (unless it's being spawned by inetd, but that's another story). If you're planning this as something you do manually on demand for testing, you could suggest people setup a nice little shell script or alias to start /usr/local/sbin/varnishd (which would include all the command-line options listed above). Or, just document it such that the command to cut and paste includes the full path to the daemon's binary.

Cheers,
-Derek (dww)

May 4, 2010 - 2:50pm Nate Haug

Ahhh, sbin. Thanks!

Thanks Derek, I wasn't aware of the purpose of sbin. I suppose that if I had installed Apache through MacPorts httpd would probably end up in there too. For some reason Memcache ends up in /usr/bin/memcached instead of an sbin directory, though I'm not sure if that "d" is intended to be "daemon" or if that's just the name of the file. The httpd is in a completely oddball place for MAMP users (/Applications/MAMP/Library/bin/httpd).

For our purposes, we manually start and stop varnish as necessary for testing. In such a scenario, it sounds like moving it to the normal bin directory could be feasible. Using sbin as you've described certainly makes more sense for the seasoned administrator and in true server configurations.

devmatt (not verified) on May 4, 2010 - 2:40am

MAMP Pro 51% off at MacUpdate Promo

http://www.mupromo.com/deal/1184/5801

Great timing if you ask me :D

xtfer (not verified) on May 5, 2010 - 12:26am

Errors on ./configure

I'm getting the following error when trying to run ./configure...

./configure: line 12386: PKG_PROG_PKG_CONFIG: command not found
./configure: line 12388: syntax error near unexpected token `PCRE,'
./configure: line 12388: `   PKG_CHECK_MODULES(PCRE, libpcre)'

The first time I ran autogen it told me pkg-config was not installed. The second time it ran fine.

(This is on 10.6.3, with a clean Macports install)

Anyone got any ideas?

Eric (not verified) on June 15, 2010 - 6:19am

I needed to install pcre

You can run:
sudo port install pcre

This fixed a similar error message for me. I also needed to close my terminal and open a new one after installing pkg-config.

adrian (not verified) on May 15, 2010 - 2:55am

I'm same on you, xtfer

I'm same on you, xtfer

Anonymous (not verified) on May 31, 2010 - 3:36am

nice article

hi,
Really it is nice , it helped me a lot

Geoff B (not verified) on June 17, 2010 - 2:59pm

install pkg-config

While installing on 10.6.5, I also needed to run:

sudo port install pkgconfig

before Step 3.

Geoff B (not verified) on June 17, 2010 - 4:13pm

10.6.4, not 10.6.5 (i'm

10.6.4, not 10.6.5 (i'm installing... into the future!)

josh koenig (not verified) on July 12, 2010 - 6:49pm

Varnish Drupal module takes PHP sockets support

If you're a fan of Acquia's DAMP installer and trying to do varnish development with my Drupal module, you may find the lack of sockets support annoying.

Here's how to fix that. ;)

James Sansbury on July 16, 2010 - 3:40pm

Install Varnish with MacPorts or Homebrew

It looks like you can just install Varnish straight from MacPorts or Homebrew:

$ sudo port install varnish

OR:

$ sudo brew install varnish

Xerox Phaser Ink (not verified) on July 17, 2010 - 1:04am

Thanks for sharing, I will

Thanks for sharing, I will install and report the status.

Bryan

About this 'bot

Nate Haug

Nate Haug adds a dash of design to Lullabot. He received degrees in both a Fine Arts and Computer Science from Truman State University, creating the perfect bridge between the technical and aesthetic. Detail is his obsession, so if you know what you want, Nate will deliver your desire.

Nate joined Drupal development in 2005,...

more

Recent

Drupal Voices 160: Moshe Weitzman on Page Rendering in Drupal 7

Podcast 9.02.2010

Drupal Voices 159: John Albin Wilkins on Drupal 7 Theming

Podcast 9.01.2010

Drupal Voices 158: Emma Jane Hogbin on PHP for Designers

Podcast 8.31.2010

Command Line Basics: More Editing with Vi/Vim

Video 8.31.2010

Lullabot's Back to School Sale

Blog 8.30.2010

Popular

Drupal Voices 160: Moshe Weitzman on Page Rendering in Drupal 7

Podcast 9.02.2010

Drupal Voices 159: John Albin Wilkins on Drupal 7 Theming

Podcast 9.01.2010

Drupal Voices 158: Emma Jane Hogbin on PHP for Designers

Podcast 8.31.2010

Installing Memcached on RedHat or CentOS

Article 8.20.2009

Photo galleries with Views Attach

Article 6.01.2009
 
  • Home
  • Services
  • Events
  • Ideas
  • Store

Connect the Bots:

Twitter Facebook YouTube blip.tv All Posts Newsletter
  • Ideas
  • Blog
  • Podcasts
  • Videos
  • About
  • Contact
  • Jobs
  • Services
    • Training
  • Events
    • Training Workshops
    • Other Events
    • Conferences
    • Calendar
  • Products
    • Videos
    • Books
    • Swag
  • Ideas
    • Blog
    • Podcast
    • Videos
  • About
    • Philosophy
    • Team
    • Presskit
  • Contact
    • General
    • Work Inquiries
    • Mailing List