Lullabot Ideas
We know stuff. We empower you to know stuff too.
Module Monday: Entity Cache
Article by James SansburyOctober 24, 2011 - 7:30pm
Making Entities Scalable and Performant
Most of us are well aware about Drupal's latest foray into abstract nouns: Entities. An entity is any sort of content on your site, be it a node, user, comment, etc. If you're noticing that loading up nodes, users or comments has become a bottleneck on your site, thanks to the Entity Cache module, scaling this burden out to a separate caching layer can be as simple as a click of a button. The Entity Cache module stores the fully loaded object into cache using Drupal's cache API. This means that when that object is requested again, it can grab the whole object out of that cache, bypassing expensive SQL queries and other resource intensive tasks.
This module is lean and mean and needs no administration interface, so to showcase it, I set up a Drupal site and ran some quick benchmarks against it. To start, I installed a fresh Drupal 7 site and generated 1000 users using the Devel Generate module that comes packaged with Devel. I then wrote and enabled a quick custom module that mimics what you might see if your site executes some expensive operations when a user is loaded. I also created this simple Drush script to load up each of those 1000 users and then display the total number of seconds or milliseconds it took to process:
<?php
$start = microtime(TRUE);
for ($i = 1; $i <= 1000; $i++) {
if (user_load($i)) {
$loaded++;
}
}
$duration = microtime(TRUE) - $start;
// If it's less than 1 second, display in milliseconds.
if ($duration < 1) {
$time = ceil($duration * 1000);
$granularity = 'milliseconds';
}
// Otherwise, use seconds.
else {
$time = ceil($duration);
$granularity = 'seconds';
}
$dt_args = array(
'!count' => $loaded,
'!time' => $time,
'@granularity' => $granularity,
);
drush_log(dt('!count users were loaded in !time @granularity.', $dt_args), 'ok');
?>So let's go try this script out and see how long it takes.
$ drush scr entitycachetest.php
1000 users were loaded in 39 seconds. [ok]Drupal comes with it's own caching layers, so let's run that script a few more times to see how long it takes with those caches filled up.
$ drush scr entitycachetest.php
1000 users were loaded in 16 seconds. [ok]
$ drush scr entitycachetest.php
1000 users were loaded in 16 seconds. [ok]Ok, so it looks like it's now executing in about 16 seconds pretty consistently. Now let's download and enable the Entity Cache module and compare.
$ drush dl entitycache
Project entitycache (7.x-1.1) downloaded to [success]
drupal7/sites/all/modules/contrib/entitycache.
$ drush en entitycache -y
The following extensions will be enabled: entitycache
Do you really want to continue? (y/n): y
entitycache was enabled successfully. [ok]Ok, we've downloaded and enabled Entity Cache module, so let's fire off our script once more.
$ drush scr entitycachetest.php
1000 users were loaded in 43 seconds. [ok]Hmm, so at first it would appear that our performance actually degrades. Keep in mind, these caches need to get filled up first, so let's run it again and see what kind of performance we get.
$ drush scr entitycachetest.php
1000 users were loaded in 558 milliseconds. [ok]Whoa! Talk about a performance increase! We went from taking 16 seconds to load 1000 users to less than 1 second. We paid for it a bit in the beginning, as it took an additional 4 seconds the first time the script was run, but I think it was worth it!
One thing to note here is that I'm not using Memcache or any other caching plugin, so the Entity Cache module will be storing all this stuff in MySQL. This means if you are using Memcache, you'll probably see even better results.
At the time of this writing Entity Cache module has support for node, comment, taxonomy term, taxonomy vocabulary, file and user entities. There is currently no support for non-core entities, however it is possible to write your own integration. It has a stable release for Drupal 7, but will not be back ported for Drupal 6. If you are looking for similar solutions for Drupal 6, the maintainer suggests looking to this Pressflow merge request or the Advanced cache module.
If your Drupal 7 site is slow, lots of things can contribute to that, so I urge you not to just download and enable every module that boasts performance benefits. It's best to isolate the bottleneck first, and then find solutions tailored for that bottleneck. That said, if you find that loading of certain entities is that bottleneck, this module is a quick win to not only make your site more performant, but also be able to scale the amount of traffic your site can receive by redirecting database traffic to Memcache instead.
About this 'bot
Prior to joining Lullabot, James was lead Drupal developer at Sprocket in Atlanta, GA since 2007. Sprocket, a design-focused, web-development shop has been building sites for the past 6 years, but James helped them to standardize their development with Drupal and helped them build many beautiful Drupal sites. James has contributed to Drupal core development and also released several contrib modules...
Comments
Separate page requests
So if my concepts for drupal 7 are correct then, this entity cache is only useful for a particular page request. Correct? In other words if I request separate web page with the same objects, then drupal has to load all the objects all over again.
Perhaps you're thinking of drupal_static()?
You may be thinking of
drupal_static(), as it will store things into memory just for a single page request. Drupal's Cache API will store things in a persistent cache for as long as you tell it to. By default the item will never be removed unless explicitly told to using cache_clear_all() with a cache ID. Here's the documentation on that, if you're interested. :)Aahh...
You got me..thats right!! Thanks
howto find bottlenecks in drupal 7
My drupal 7 site could be running faster but how do i find the bottlenecks
That's a BIG question
I wish I had a nice short and sweet answer to that for you, but unfortunately it's not something one can have a quick answer to. :) There are lots of reasons your site could be slow, including misconfigurations with Apache, MySQL, improper use of MemCache or APC, slow queries, tables missing indexes, and sometimes it's a matter of just not having the right hardware for the job.
One thing you could look to is the Drupalize.me video on Performance and Scalability. In that video we start from the ground up and look at every layer of the LAMP stack to optimize it for Drupal. That video is tailored to Drupal6 (Pressflow), but all of the concepts translate to Drupal 7 pretty easily.
Another thing you could investigate is enabling the Performance Logging module that comes packaged with Devel. Make sure you have it set up to use APC to store it's log, and also follow the instructions for setting it up for a production environment. It will log statistics about different pages and let you know when certain pages have problems.
Never heard about this cache...
... thanks a lot, i`ve worked a long time with drupal but until now i never have heard about entity cache. Sounds very good and will stop soon some problems with elder websites. I`ll try it out and post my feedback after my doings.
Best regards
Marcus
from Webdesign München
Entity Cache Module
The the Entity Cache module has been a huge relief for my programmers. We deal with a lot of Indianapolis Web Design clients, and this has been a huge time saver for us. It has bee so nice that the Entity Cache module stores the fully loaded object into cache using Drupal's cache API. I love this module!
Entitycache and filecache
I installed filecache and entitycache in a one site and i use jmeter for the performance's test. Here the result : firt filecache , second entitycache and last the combinasion of the two module. My question is why the result is that??