[Update 2] – It seems W3 Total Cache support has been silently discontinued – it’s not been updated for over 12 months now, and has allegedly got some security issues. Because of this, I would recommend not installing W3 Total Cache as part of these instructions, everything else should run fine. [/Update]



[Update] – I’ve tested this using the new Ubuntu 12.04 LTS Edition, and the settings all appear to work correctly without modification. Let me know if you find a problem [/Update]

[Update September 2013] – I’ve recently moved my own blog hosting to a $10 a month VPS from Digital Ocean (that link gets you a $10 discount via an affiliate link) and have been very happy with them, including the performance of the blog.



These instructions are the rather verbose, but hopefully easy enough to follow, steps to build a new Linux server using Varnish, Nginx, W3 Total Cache, and WordPress, to build a WordPress blog on a Amazon Micro server (or equivalent), all costing under $15 a month, capable of sustaining 10 million hits per day, as measured by blitz.io.

10 Million hits per day with WordPress on a $15 virtual server

Install Ubuntu 11.10 (Oneiric) on a new virtual private server- it needs to be 11.10 for all the packages that I’m going to list to work out of the box, but it’s all possible to do with other distributions.

I used Amazon EC2 to build my test server, but Linode are also very good. As of September 2013, this blog is now hosted on Digital Ocean

For the purpose of the documentation, my server details were as follows, yours will be different:

Public DNS Name: ec2-23-20-235-223.compute-1.amazonaws.com

Public IP Address: 23.20.235.223

Login to the server and become root



login as ubuntu via ssh, then run the sudo -i command to become root

ssh ubuntu@ec2-23-20-235-223.compute-1.amazonaws.com

sudo -i

Configure a firewall first



Since we’re going to be installing various network services which by default listen on all interfaces, it’s important to configure a firewall.

For Ubuntu, this is nice and easy, simply use the ufw package.

Execute the following commands to configure it:

ufw allow ssh

ufw allow http

ufw logging off

ufw enable

Once this is done, your server has a relatively secure firewall, though it’s worth looking at fail2ban to prevent brute force password attacks.

If you’re using Amazon EC2, you’ll also need to open the Security Group to allow traffic on port 80. You can do this using the AWS Security Groups Console, you might need to change the region. Select the security group you used when you started the instance, and then click “Inbound”, then select “HTTP” from the drop down menu, then finally click “Add Rule”. You don’t need to restart the instance for it to take effect.

Install and Configure MySQL



apt-get update

apt-get install mysql-server

When prompted, set a mysql “root” user password

mysql -u root -p

When prompted, enter your newly set root password

At the mysql> prompt, run the following 4 commands, replacing ENTER_A_PASSWORD with a password of your own

CREATE DATABASE wordpress;

GRANT ALL PRIVILEGES ON wordpress.* TO “wp_user”@”localhost” IDENTIFIED BY “ENTER_A_PASSWORD”;

FLUSH PRIVILEGES;

EXIT

That’s MySQL installed, ready for the PHP and Web server installation (nginx).

Install and configure PHP



We need to install not just PHP, but the PHP FPM system, APC, and the MySQL module

apt-get install php5-fpm php-pear php5-common php5-mysql php-apc php5-gd

Edit /etc/php5/fpm/php.ini and add these lines at the bottom:

[apc]

apc.write_lock = 1

apc.slam_defense = 0

Edit /etc/php5/fpm/pool.d/www.conf

Replace

listen = 127.0.0.1:9000

with

listen = /dev/shm/php-fpm-www.sock

Below that, insert these 3 lines

listen.owner = nginx

listen.group = nginx

listen.mode = 0660

Then, further down in the same file, replace these 2 lines

user = www-data

group = www-data

with

user = nginx

group = nginx

Save the file, PHP FPM is now complete, but it won’t work until we install nginx, so don’t worry about starting it now.

Install and Configure Nginx



Instructions based on the Nginx website.

Download the nginx secure key to verify the package

cd /tmp/

wget http://nginx.org/keys/nginx_signing.key

apt-key add /tmp/nginx_signing.key

Add the sources to the APT sources file by running these 2 commands (the >> is important!)

echo “deb http://nginx.org/packages/ubuntu/ lucid nginx” >> /etc/apt/sources.list

echo “deb-src http://nginx.org/packages/ubuntu/ lucid nginx” >> /etc/apt/sources.list

Download and install nginx by running

apt-get update

apt-get install nginx

When that completes, nginx will be installed, but needs configuring for WordPress.

nginx configuration files are in /etc/nginx

First, edit /etc/nginx/nginx.conf

Inside the http section , insert the following lines so that when you later add varnish in front, things don’t break all over the place, and so varnish caches compressed files:

port_in_redirect off;

gzip on;

gzip_types text/css text/xml text/javascript application/x-javascript;

gzip_vary on;

Next, cd to /etc/nginx/conf.d and create a new file, /etc/nginx/conf.d/drop with the contents of the drop file from GitHub

Then, replace /etc/nginx/conf.d/default.conf with the contents of the github default.conf file changing all entries for domainname.com with your own domain name (there’s 3 entries, including 1 near the bottom)

Make a directory, /var/www/ and set the ownership:

mkdir -p /var/www/

chown nginx:nginx /var/www/

chmod 775 /var/www

That’s nginx configured, restart it and the PHP FPM service by running:

service nginx restart

service php5-fpm restart

Now, you’re actually ready to install WordPress!



This is pretty simple, run:

cd /tmp

wget http://wordpress.org/latest.tar.gz

tar zxvf latest.tar.gz

cd wordpress

mv * /var/www/

chown -R nginx:nginx /var/www

To configure WordPress, run:

cp /var/www/wp-config-sample.php /var/www/wp-config.php

chown nginx:nginx /var/www/wp-config.php

In a web browser, visit https://api.wordpress.org/secret-key/1.1/salt/ and copy the results

Edit /var/www/wp-config.php

and scroll down to fine the AUTH_KEY line down to NONCE_SALT, and replace them with the values you copied from the api.wordpress.org site

Then, replace the default values with the MySQL ones you chose earlier (not the root user):

define(‘DB_NAME’, ‘database_name_here’);

define(‘DB_USER’, ‘username_here’);

define(‘DB_PASSWORD’, ‘password_here’);

And once it’s done, if you’ve not had any errors and your domain name is pointing at the right IP (this is important!), then you’ll be able to visit your domain and see the WordPress configuration page at http://www.domainname.com/wp-admin/install.php

Go through install questions, choose a sensible username and password (it’s highly recommended you change the admin user from “admin” to something else.

Go to settings then select permalinks, and choose “Custom Structure”, and paste in the value below (including the % symbols)

/%post_id%/%postname%

Then hit “Save Changes”

It’s time to create a test post, so click on “Posts” then “Add New”

Enter a title and body, then hit “Post”, and make a note of it’s friendly URL.

Run a blitz.io test to see how we’re doing

100 users, 60 seconds. Timeouts, low hit rates, errors, etc. CPU flat out, seems to be the initial bottleneck.

This rush generated 632 successful hits in 1.0 min and we transferred 1.76 MB of data in and out of your app. The average hit rate of 9.81/second translates to about 847,776 hits/day.

You got bigger problems though: 34.91% of the users during this rush experienced timeouts or errors!

So the server is running, but it’s still too slow!

Next, we will enable the WordPress caching systems

Go to the wordpress admin page, then plugins, and click install new plugin.

Update Skip the W3 Total Cache plugin if you’re following this as a tutorial, it’s not been updated in ages. I’ll update this post in the future with W3 Super Cache instructions instead, but everything else should work as it is for now

Search for “W3 Total Cache”, then click “Install Now” when the search results return. When installation is complete, click “Activate Plugin”.

Go to the new “Performance” section in the menu at the left side of the page.

Scroll through the cache options, selecting “PHP APC” at each opportunity and enabling the following 2 sections :

Database Cache

Object Cache

Hit “Save All Settings” then hit “Deploy”

Rerun the blitz.io test again, performance should be much improved

This rush generated 2,902 successful hits in 1.0 min and we transferred 27.59 MB of data in and out of your app. The average hit rate of 46/second translates to about 4,006,008 hits/day.

If we then up the blitz.io run from 100 users to 250 users, there are still problems we need to fix:

This rush generated 4,733 successful hits in 1.0 min and we transferred 36.20 MB of data in and out of your app. The average hit rate of 72/second translates to about 6,297,653 hits/day.

You got bigger problems though: 5.49% of the users during this rush experienced timeouts or errors!

You can see the Blitz results from the Nginx And APC configuration in this PDF.

So still not perfect – Time to install varnish 3!

apt-get install varnish

Edit /etc/varnish/default.vcl replace the contents with the file default.vcl file from github.

Edit /etc/default/varnish

Change the section

DAEMON_OPTS=”-a :6081 \

-T localhost:6082 \

-f /etc/varnish/default.vcl \

-S /etc/varnish/secret \

-s malloc,256m”

to

DAEMON_OPTS=”-a :80 \

-T localhost:6082 \

-f /etc/varnish/default.vcl \

-S /etc/varnish/secret \

-s malloc,64m”

Next, we need to edit the nginx configuration to listen on port 8080, instead of port 80 – Varnish is going to be running on port 80 instead.

Edit /etc/nginx/conf.d/default.conf and replace

listen 80;

with

listen 8080;

Save the file, then run

service nginx restart

service varnish restart

Re-run the blitz.io test with 100 users



Almost no CPU usage, should get perfect results

This rush generated 2,959 successful hits in 1.0 min and we transferred 28.36 MB of data in and out of your app. The average hit rate of 47/second translates to about 4,093,000 hits/day.

Re-run the blitz with 250 users



This rush generated 7,342 successful hits in 1.0 min and we transferred 70.38 MB of data in and out of your app. The average hit rate of 117/second translates to about 10,134,627 hits/day.

You can see the full final run blitz performance details in this PDF.

There it is, 10 million hits per day using WordPress on a box costing less than $20 a month, all thanks to varnish and nginx, easy!

Remember to get your $10 discount from Digital Ocean.