How to install and configure Varnish cache on Ubuntu Linux 16.04 LTS

ADVERTISEMENTS



Our setup

How to install and configure Varnish cache

How do I install Varnish Cache server to improve the performance of my existing Apache web server especially caching dynamic content on Ubuntu Linux 16.04 LTS server?Varnish Cache is a web application accelerator. You install it in front of any web-server that speaks HTTP and configure it to cache the contents. It act as a web application accelerator focusing on optimizing caching and compression It is really fast and used by high traffic websites. Let us see

Type the following apt-get command/apt command to install Varnish cache software:

$ sudo apt update

$ sudo apt upgrade

$ apt search Varnish

$ sudo apt install varnish

Sample outputs:

Reading state information... Done The following additional packages will be installed: binutils cpp cpp-5 gcc gcc-5 libasan2 libatomic1 libcc1-0 libcilkrts5 libgcc-5-dev libgomp1 libisl15 libitm1 libjemalloc1 liblsan0 libmpc3 libmpx0 libquadmath0 libtsan0 libubsan0 libvarnishapi1 Suggested packages: binutils-doc cpp-doc gcc-5-locales gcc-multilib make autoconf automake libtool flex bison gdb gcc-doc gcc-5-multilib gcc-5-doc libgcc1-dbg libgomp1-dbg libitm1-dbg libatomic1-dbg libasan2-dbg liblsan0-dbg libtsan0-dbg libubsan0-dbg libcilkrts5-dbg libmpx0-dbg libquadmath0-dbg varnish-doc The following NEW packages will be installed: binutils cpp cpp-5 gcc gcc-5 libasan2 libatomic1 libcc1-0 libcilkrts5 libgcc-5-dev libgomp1 libisl15 libitm1 libjemalloc1 liblsan0 libmpc3 libmpx0 libquadmath0 libtsan0 libubsan0 libvarnishapi1 varnish 0 upgraded, 22 newly installed, 0 to remove and 0 not upgraded. Need to get 23.1 MB of archives. After this operation, 79.7 MB of additional disk space will be used. Do you want to continue? [Y/n] y Get:1 http://in.archive.ubuntu.com/ubuntu xenial/main amd64 libmpc3 amd64 1.0.3-1 [39.7 kB] ..... ... .. Setting up libquadmath0:amd64 (5.4.0-6ubuntu1~16.04.4) ... Setting up libgcc-5-dev:amd64 (5.4.0-6ubuntu1~16.04.4) ... Setting up gcc-5 (5.4.0-6ubuntu1~16.04.4) ... Setting up gcc (4:5.3.1-1ubuntu1) ... Setting up libvarnishapi1 (4.1.1-1) ... Setting up libjemalloc1 (3.6.0-9ubuntu1) ... Setting up varnish (4.1.1-1) ... Processing triggers for libc-bin (2.23-0ubuntu5) ... Processing triggers for systemd (229-4ubuntu16) ... Processing triggers for ureadahead (0.100.0-19) ... Reading state information... Done The following additional packages will be installed: binutils cpp cpp-5 gcc gcc-5 libasan2 libatomic1 libcc1-0 libcilkrts5 libgcc-5-dev libgomp1 libisl15 libitm1 libjemalloc1 liblsan0 libmpc3 libmpx0 libquadmath0 libtsan0 libubsan0 libvarnishapi1 Suggested packages: binutils-doc cpp-doc gcc-5-locales gcc-multilib make autoconf automake libtool flex bison gdb gcc-doc gcc-5-multilib gcc-5-doc libgcc1-dbg libgomp1-dbg libitm1-dbg libatomic1-dbg libasan2-dbg liblsan0-dbg libtsan0-dbg libubsan0-dbg libcilkrts5-dbg libmpx0-dbg libquadmath0-dbg varnish-doc The following NEW packages will be installed: binutils cpp cpp-5 gcc gcc-5 libasan2 libatomic1 libcc1-0 libcilkrts5 libgcc-5-dev libgomp1 libisl15 libitm1 libjemalloc1 liblsan0 libmpc3 libmpx0 libquadmath0 libtsan0 libubsan0 libvarnishapi1 varnish 0 upgraded, 22 newly installed, 0 to remove and 0 not upgraded. Need to get 23.1 MB of archives. After this operation, 79.7 MB of additional disk space will be used. Do you want to continue? [Y/n] y Get:1 http://in.archive.ubuntu.com/ubuntu xenial/main amd64 libmpc3 amd64 1.0.3-1 [39.7 kB] ..... ... .. Setting up libquadmath0:amd64 (5.4.0-6ubuntu1~16.04.4) ... Setting up libgcc-5-dev:amd64 (5.4.0-6ubuntu1~16.04.4) ... Setting up gcc-5 (5.4.0-6ubuntu1~16.04.4) ... Setting up gcc (4:5.3.1-1ubuntu1) ... Setting up libvarnishapi1 (4.1.1-1) ... Setting up libjemalloc1 (3.6.0-9ubuntu1) ... Setting up varnish (4.1.1-1) ... Processing triggers for libc-bin (2.23-0ubuntu5) ... Processing triggers for systemd (229-4ubuntu16) ... Processing triggers for ureadahead (0.100.0-19) ...

Varnish installed and it is configured to listen on port 6081. This is default. You can verify it with the following simple netstat command:

$ netstat -tulpn | grep varnishd

Sample outputs:



Varnish configuration on Ubuntu Linux

Ubuntu Linux 16.04 LTS uses systemd based config files as follows:

/lib/systemd/system/varnish.service – Default config file

TCP 6081 port – Varnish cache port

TCP 6082 port – Varnish management port

/etc/varnish/ – Varnish configuration files including VCLs

Let us see how to configure as per setup #1 (see fig.01).

Step 1 – Put Varnish cache on port 80

Type the following command to edit varnish cache file and create a new config file /etc/systemd/system/varnish.service.d/:

$ sudo systemctl edit varnish.service

Add/append the following config options:

[Service] ExecStart= ExecStart=/usr/sbin/varnishd -j unix,user=vcache -F -a 202.54.2.50:80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc, 1024m

Save and close the file. Where,

202.54.2.50:80 – Listen to public IP address on port 80. 1024m – Increase RAM usage for Varnish from default 256m to 1024m

Type the following command to reload changes:

$ sudo systemctl daemon-reload

Step 2 – Configure Varnish to use our Apache server as a backend

Next, you will configure Varnish to use our Apache server as a backend which is running on IP address 192.168.1.6 port 80. Edit the /etc/varnish/default.vcl, enter:

$ sudo vi /etc/varnish/default.vcl

Set the following to point to your content server (Apache/Lighttpd/PHP/Python app server) running on 192.168.1.6 port 80.

backend default { .host = "192.168.1.6" ; .port = "80" ; } backend default { .host = "192.168.1.6"; .port = "80"; }

Basically, you need to set the values of host and port match your LAMP server private IP address and listening port. Save and close the file.

Step 3 – Configure Apache to use run on IP address 192.168.1.6 port 80

Type the following command to edit the /etc/apache2/ports.conf:

$ sudo vi /etc/apache2/ports.conf

Edit/add/append Listen directive:

Listen 192.168.1.6: 80 Listen 192.168.1.6:80

Save and close the file. Edit /etc/apache2/sites-enabled/000-default.conf, enter:

$ vi /etc/apache2/sites-enabled/000-default.conf

The Virtual Host should be updated as follows :

<VirtualHost 192.168.1.6:80>

Save and close the file. Finally, restart the Apache web-server, run:

$ sudo systemctl restart apache2.service

At this stage you need to restart the Varnish cache too, enter:

$ sudo systemctl restart varnish.service

Step 4 – Configure ufw to open port 80

Type the following command to open port 80 using ufw:

$ sudo ufw allow 80

Sample outputs:

Rule added Rule added (v6)

Step 5- Test it

Fire your webbrowser and type the url or IP address:

http://your-ip-here/ http://202.54.2.50/ http://your-domain-name/

Sample outputs:



$ url -I http://202.54.2.50/

HTTP/1.1 200 OK Date: Wed, 22 Feb 2017 11:59:38 GMT Server: Apache/2.4.18 (Ubuntu) Last-Modified: Wed, 22 Feb 2017 11:42:57 GMT Vary: Accept-Encoding Content-Type: text/html X-Varnish: 32773 32771 Age: 4 Via: 1.1 varnish-v4 ETag: W/"2c39-5491cff7c5c13-gzip" Accept-Ranges: bytes Connection: keep-alive

Another option is to use the curl command to see http headers:Sample outputs:

The value of Age: indicate that pages has been cached by Varnish server.

How do I see the Varnish log and other info?

To see Varnish log entry ranking, enter:

$ sudo varnishtop

Sample outputs:



$ cd /var/log/varnish

$ sudo tail -f varnish.log

$ sudo tail -f varnishncsa.log

151.1.xx.yy - - [22/Feb/2017:12:03:48 +0000] "GET http://www.cyberciti.biz/faq/ubuntu-linux-start-restart-stop-apache-web-server/ HTTP/1.0" 200 60944 "https://www.google.co.in/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:48.0) Gecko/20100101 Firefox/48.0" 111.z.xx.yy - - [22/Feb/2017:12:03:48 +0000] "GET http://www.cyberciti.biz/faq/linux-disk-format/ HTTP/1.0" 200 94830 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" xxx.ttt.zz.tt - - [22/Feb/2017:12:03:48 +0000] "GET http://www.cyberciti.biz/faq/wp-content/plugins/contextual-related-posts/default.png HTTP/1.0" 200 15454 "https://www.cyberciti.biz/faq/ubuntu-linux-start-restart-stop-apache-web-server/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:48.0) Gecko/20100101 Firefox/48.0" aa.bb.cc.dd - - [22/Feb/2017:12:03:48 +0000] "GET http://www.cyberciti.biz/faq/find-linux-distribution-name-version-number/ HTTP/1.0" 200 92470 "https://www.google.co.in/" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0"

To see log file cd to /var/log/varnishSample outputs (provided that you configured PHP/Python with Apache you should see something as follows):

See the Varnish server cache statistics

Type the following command:

$ sudo varnishstat

$ sudo varnishstat -1

$ sudo varnishstat -1 | grep -i cache

Sample outputs:

MAIN.cache_hit 334059 6.33 Cache hits MAIN.cache_hitpass 239 0.00 Cache hits for pass MAIN.cache_miss 94300 1.79 Cache misses

How do I purge content from the Varnish cache?

First you need to define ACL. I have the following config in my /etc/varnish/default.vcl file:

# Who is allowed to purge? acl purge { "localhost"; "127.0.0.1"; "72.14.190.12"; "192.168.1.6"; } sub vcl_recv { # allow PURGE from localhost,72.14.190.12 and 192.168.1.6 if (req.method == "PURGE") { if (!client.ip ~ purge) { return(synth(405,"Not allowed.")); } return (purge); } }

Use the following command to invalidate https://www.cyberciti.biz/faq/bash-for-loop/ page I would call out to Varnish like this:

$ varnishadm -T 127.0.0.1:6082 -S /etc/varnish/secret ban "req.http.host == www.cyberciti.biz && req.url == /faq/bash-for-loop/"

To remove/purge all .html pages from cache, enter:

$ varnishadm -T 127.0.0.1:6082 -S /etc/varnish/secret "ban req.http.host ~ www.cyberciti.biz && req.url ~ .html"

If you want to purge everything just restart the Varnish cache:

$ sudo systemctl restart varnish.service

See HTTP Purging for more info. Stay tunned for more Nginx SSL config and creating custom VCL for caching pages.

See also