Drupal performance

Submitted by Peter on Tue, 2010-07-06 22:45

Drupal:

Drupal performance is dependent on the modules you select and the options within each module. Everything else is cured by the rapid increase in hardware performance.

Costs

Your first small experimental public Web site will be on shared hosting and you will not control the configuration of the Web server used for your Web site. When you step up to a money making web site and want to look at performance, you should be using at least a VPS (Virtual Private Server) when you have the type of control you need to achieve improvements in performance. Your minimum monthly cost will jump from US$8 up to US$20.

(If you use Hostmonster, as advertised at the top right corner of this page, your cost drops to US$6.95 a month and you gain more control, giving you an intermediate step before leaping up to a VPS.)

Processor usage

The most expensive part of Web site hosting is usually processor performance. You can cut processor usage by 95 percent with an op-code cache. APC and eAccelerator are two op-code caches. You do not have to know the fine details of what they do other than the fact that they use memory to save completed work for reuse without additional processor overhead. cPanel and equivalent Web hosting account control panels have a spot where you can switch on APC or eAccelerator. Other Web hosting services might force you to ask your hosting company to switch on the cache. (Some of the lower priced Web hosting accounts cost you a lot of money in the form of wasted time and work.)

You might need more memory to run the cache. Memory decreases in price faster than processing power. Based on testing across over 200 web sites, you need double the memory for small Web sites and only a trivial increase in memory for a large site. A simple site might use 16 MB per Web page delivery process and might peak at 10 pages delivered in parallel, fitting in 256 MB of memory at the hosting level. The op-code cache adds only 5 MB to 10 MB but that might be enough to bump your site from a 256 MB hosting account up to a 512 MB account. In return the cache might reduce processor usage by 20 or 40 percent.

A complex site might use 60 MB per page process and overlap the delivery of 100 pages at a time, requiring 6 GB of memory. The op-code cache adds only 5 MB to 10 MB and it might reduce each of those 100 page serving processes by 3 MB. 100 time 3 MB is 300 MB saved, far more than the 10 MB used by the cache. You also use less processor time per page.

Two years ago there were a lot of people offering 128 MB VPS accounts. Today they all offer a minimum of 256 MB or greater. The memory for an op-code cache is not an issue.

A typical $20 account gives you 2.5 percent of a medium size processor. A $50 account gives you 10 percent of the same processor. An op-code cache will often drop your usage from a $50 account to a $20 account.

Memory usage

Memory is required for the Web server and the database behind the Web site. If the database is running in the same physical server as the Web server, you need a lot of memory. When the database has a lot of memory and is configured to use it, the database saves query results in memory and common queries are served from memory, not disk. The database also caches indexes and other things in memory. A database starved of memory will constantly reread all that data from disk, slowing your Web site down to a trickle.

One of the first expansions from one server to two servers is to move the database to a separate server. The Web server can then hog all the memory on one server and the database can feast on all the memory in the other server. If you have not updated your server for a couple of years, you can usually double the available memory by just replacing your old server with a current model fitted with the maximum configurable memory, saving you the expansion to two servers.

Good performance depends on lots of memory. Good performance depends on all that memory being used. There are a whole lot of configuration settings in your Web server, usually Apache, and your database server, usually MySQL. Many of those configuration settings increase memory usage. If the settings are not right, you could buy a lot of memory and use only a little bit. You could also over allocate memory and have Apache fighting MySQL for non existent memory. You want someone to check the Apache and MySQL settings before you put a lot of work into changing Drupal.

Drupal memory usage depends on the modules loaded. If you load a module then decide not to use the module, switch the module off and delete the module. You have just made your first memory usage reduction. Are there any other modules you could delete?

Drupal stores a lot of things in the database. Drupal saves some overheads by storing completed results in the database then delivering the saved results. The Drupal caching process saves processing time and the database overheads previously used to repeatedly generate the results. To get maximum benefit from the cache, your database needs lots of memory to quickly retrieve the saved results.

Check the Drupal performance settings in the Administration section to ensure Drupal is making the best use of the Drupal cache facilities.

If the database has enough memory, frequently used data will be retained in memory and reused. When you make changes to Drupal settings or add modules, you may need to review the database memory allocations.

Given the low cost of memory, buy heaps of memory, track memory usage, and buy more when there is the slightest shortage.

Module selection

You select modules for functionality. You often get the choice of several modules to provide similar functionality. The choice can alter performance. Comparing two modules, one might add no database processing overheads while the other might add a couple of extra database tables and at least one access to each table for every Web page delivered. Your existing Web site might use 30 database table accesses to deliver a page and you add an extra two, creating a slight performance decrease. Another site might already use 100 database accesses per page and an extra two might not be noticed.

Not all modules work the same. An add-on module might add an extra 200 database accesses per page, an amount that would kill the performance of a site previously using only 30 accesses per page. You will find a few performance recommendations by me at drupal.org for Drupal 7, 6, 5, and 4. They are not rocket science, just experience coupled with a logical approach to analysis and testing. Lots of people are trained to analyse. Not many people understand testing and few people in the Drupal area have the extensive database experience required for performance improvements. The good thing is the pool of 800,000 people contributing to Drupal. Some of them have the right combination to improve the database performance of the more popular modules. When you are choosing between several modules for the same task, look at the number of people using the modules and their comments about performance.

Are your visitors logged in?

When you first visit a Drupal based Web site, you are an anonymous user and received cached Web pages. A cached Web page might be refreshed only once every 5 minutes or more. You log in then become a user connected to an account. Logged in users get fresh pages all the time. Logged in users use more resources because their Web pages are rebuilt for every page view.

100 logged in users can easily use more resources than 1000 anonymous visitors. A test at one Web site showed 1000 logged in users using more resources than 50,000 anonymous users. You can see how misleading it would be to tune your Web site for anonymous users if most of your visitors log in.

If your Web site is oriented to logged in users, buy more processor power, a lot more memory, and allocate a lot of memory to the database. Perform all your testing and tuning with lots of users logged in.

Some modules are only activated for logged in users. Consider forum modules that highlight unread forum entries. The module has to identify all the forum entries listed on the current page then look up a database table to find if you have read them. None of that overhead is required for anonymous visitors. When you add a new module, test the performance difference for both anonymous visitors and logged in users.

Drupal 5

Drupal 5 had some performance improvements, including SQL enhancements, developed during the lifetime of Drupal 5. Switching on extra options produced the occasional increase in overhead with some options producing awful performance. Several of the bad performing optional modules were greatly improved during the life of Drupal 5.

I contributed some of the SQL and code improvements based on years of experience with databases and making applications go faster. I demonstrated a code file split technique for improving Drupal 5 .module files and a code file splitting technique was made standard in Drupal 6. Some of the other improvements included adding options to exclude expensive features if you did not use them and improved documentation on the effect of using some options.

Drupal 6

Drupal 6 has more enhancements and better performance testing. Measurement of 50 Web site conversions from Drupal 5 to 6 showed that not one site ran slower and some ran faster. Some other Web site conversions included switching on extra options and produced the occasional increase in overhead due to the extra work performed.

Drupal 7

Drupal 7 uses more resources than Drupal 6. There is a lot of performance tuning work in Drupal 7 that gives you a big improvement in functionality from only a small increase in processor overheads. Some of that extra overhead is from modules that were optional in Drupal 6 and are included as standard in Drupal 7. Drupal 7 uses about 30 percent more processing power per page compared to Drupal 6 but most Drupal 6 sites had some of those extra modules turned on and will not see a difference.

In fact some people will see a performance improvement because they used all the previously optional modules and the modules are now more efficient in Drupal 7.

Drupal 7 has more cache options than Drupal 6. You may see a performance improvement because your site now caches something that was not previously cached.

Possible improvements

Drupal creates several tables for temporary information, mostly cached information. The tables do not have to be on disk because they are recreated automatically if the server restarts. MySQL users could change the tables to a type named memory. If your server has spare memory and you have tuned MySQL to use the maximum useful amount of memory, allocate some more to MySQL then change your table types to memory.

A more complicated approach is to use a product named memcache. Memcache effectively does the same as the MySQL memory table but using more code because you have to replace the standard Drupal cache code with code for memcache. I do not recommend adding memcache to your Web site without practicing on a similar Web hosting account. There are too many things to go wrong.