Partial pages are nice, but being able to write in markdown would be even better. Cegdown is a Clojure wrapper for Pegdown, a popular Java library for rendering markdown. Add it to your project.clj:

:dependencies [ [ org.clojure/clojure "1.5.1" ] [ stasis "1.0.0" ] [ ring "1.2.1" ] [ hiccup "1.0.5" ] [ me.raynes/cegdown "0.1.1" ] ]

Now, add it to the namespace form in src/cjohansen_no/web.clj. While we're at it, we will require the Clojure string library as well (we'll use it shortly):

( ns cjohansen-no.web ( :require [ clojure.java.io :as io ] [ clojure.string :as str ] [ hiccup.page :refer [ html5 ] ] [ me.raynes.cegdown :as md ] [ stasis.core :as stasis ] ) )

Now we will add a function to render every page in resources/md as markdown. It will be very similar to the partials we did before, but now with markdown rendering as well. Because we don't want ".md" as part of the URL for these pages, we will map the keys as well.

( defn markdown-pages [ pages ] ( zipmap ( map # ( str/replace % #"\.md$" "" ) ( keys pages ) ) ( map # ( layout-page ( md/to-html % ) ) ( vals pages ) ) ) )

The #( ) form is a function literal. Inside it, % refers to the first argument. In the above example, the following are identical:

# ( str/replace % #"\.md$" "" ) ( fn [ path ] ( str/replace path #"\.md$" "" ) )

I wrote more on the anonymous function literal in a separate post.

The final step is to add the new page source to our map:

( defn get-pages [ ] ( stasis/merge-page-sources { :public ( stasis/slurp-directory "resources/public" #".*\.(html|css|js)$" ) :partials ( partial-pages ( stasis/slurp-directory "resources/partials" #".*\.html$" ) ) :markdown ( markdown-pages ( stasis/slurp-directory "resources/md" #"\.md$" ) ) } ) )

Add the following to resources/md/my-first-post.md :

# My first post It's pretty short for now.

Restart the server again (we added more new dependencies, remember?) When you have, /my-first-post should present you with your brief, but lovely blog post.

Any self-respecting tech blog needs nice syntax highlighting for code blocks. When it comes to syntax highlighting, Pygments is the bee's knees. It supports just about any language you can think of, there's a bunch of color themes around and it is stable and resillient. It is also the library used to highlight code on GitHub. Clygments is a Clojure interface to it (which uses Jython; Pygments is a Python library).

We will add syntax highlighting as a post-processing step for HTML. This way, we can support syntax highlighting for full static pages in the public directory, partial pages and pages rendered from markdown. To do this, we will use another templating library for Clojure, enlive. Actually, enlive is more than just a templating library. As you will see, it can be used to transform documents in various interesting ways.

Let's add a fenced code block to our markdown file:

# My first post It's pretty short for now. Here's our project.clj: ```clj (defproject cjohansen-no "0.1.0-SNAPSHOT" :description "cjohansen.no source code" :url "http://cjohansen.no" :license {:name "BSD 2 Clause" :url "http://opensource.org/licenses/BSD-2-Clause"} :dependencies [[org.clojure/clojure "1.5.1"] [stasis "1.0.0"] [ring "1.2.1"] [hiccup "1.0.5"] [me.raynes/cegdown "0.1.1"]] :ring {:handler cjohansen-no.web/app} :profiles {:dev {:plugins [[lein-ring "0.8.10"]]}}) ```

In order for this work, we need to inform cegdown that we want to enable the fenced code blocks extension. While we're at it, we'll enable a couple of other useful extensions as well:

( def pegdown-options [ :autolinks :fenced-code-blocks :strikethrough ] ) ( defn render-markdown-page [ page ] ( layout-page ( md/to-html page pegdown-options ) ) ) ( defn markdown-pages [ pages ] ( zipmap ( map # ( str/replace % #"\.md$" "" ) ( keys pages ) ) ( map render-markdown-page ( vals pages ) ) ) )

Reloading the blog post will show you how the fenced code blocks are rendered:

< pre >< code class = "clj" > ... </ code ></ pre >

We will now use enlive to extract this piece of markup and replace it with the version highlighted by clygments. First off, add the new dependencies to project.clj (remember to restart the server!)

:dependencies [ [ org.clojure/clojure "1.5.1" ] [ stasis "1.0.0" ] [ ring "1.2.1" ] [ hiccup "1.0.5" ] [ me.raynes/cegdown "0.1.1" ] [ enlive "1.1.5" ] [ clygments "0.1.1" ] ]

Add a new namespace (i.e., file) to the project. Copy the following code into src/cjohansen_no/highlight.clj :

( ns cjohansen-no.highlight ( :require [ clojure.java.io :as io ] [ clygments.core :as pygments ] [ net.cgrand.enlive-html :as enlive ] ) )

Enlive has many tricks up its sleave. Perhaps the most interesting one is the somewhat confusingly named sniptest . It takes some HTML as a string, and selector/function pairs. It will then;

Parse the HTML Find all nodes matching the selector Call the corresponding function once for every match Replace the node with the result of calling the function Return the transformed HTML as a string

( defn highlight-code-blocks [ page ] ( enlive/sniptest page [ :pre :code ] highlight [ :pre :code ] # ( assoc-in % [ :attrs :class ] "codehilite" ) ) )

This function will find every code element inside a pre element and pass it through the highlight function (to be shown). Then, it will add a class name to the same elements. In practice, you would probably do both in the highlight function, but this allows me to illustrate how you can perform multiple transformations in one go. The "codehilite" class name just happens to be class name used by the Pygments CSS themes available here (we will include this later). Add the second function:

( defn- highlight [ node ] ( let [ code ( ->> node :content ( apply str ) ) lang ( ->> node :attrs :class keyword ) ] ( pygments/highlight code lang :html ) ) )

The dash in defn- means that this function is private, and only referrable within the current namespace. The nodes that enlive operate on are maps like this:

{ :tag :code :attrs { :class "clj" } :content [ ... ] }

The content is a list of new nodes and/or strings.

->> is the thread-last macro. It takes any number of arguments, threads values from left to right; given (->> a (b 1 2) c) , it will take the value a , pass it as the last argument to b (i.e. (b 1 2 a) ), pass the return value of that expression as the last argument to c , and finally return the result of that. So the above line:

( ->> node :content ( apply str ) )

Is the same as this:

( apply str ( :content node ) )

I wrote more on threading macros in a separate post.

Using a keyword as a function is one way to look up that key in a map. The apply call means to call str with the following list of arguments, as individual arguments, not one list (e.g. (fn a b c) , not (fn [a b c]) ).

To preview the highlighting in the browser, we need to make some changes to the web namespace. Start by updating the namespace form to pull in our new dependency:

( ns cjohansen-no.web ( :require [ cjohansen-no.highlight :refer [ highlight-code-blocks ] ] [ clojure.java.io :as io ] [ clojure.string :as str ] [ hiccup.page :refer [ html5 ] ] [ me.raynes.cegdown :as md ] [ stasis.core :as stasis ] ) )

Syntax highlighting should apply to all pages. A good place to do it is between our old get-pages function and Stasis' rendering. We will do this by adding a prepare-pages function:

( defn prepare-pages [ pages ] ( zipmap ( keys pages ) ( map # ( highlight-code-blocks % ) ( vals pages ) ) ) )

Again, we use zipmap to produce a new map where the keys are untouched, but the values have been mapped. To make Stasis run through this, rename get-pages to get-raw-pages , and add a new get-pages:

( defn get-raw-pages [ ] ( stasis/merge-page-sources { :public ( stasis/slurp-directory "resources/public" #".*\.(html|css|js)$" ) :partials ( partial-pages ( stasis/slurp-directory "resources/partials" #".*\.html$" ) ) :markdown ( markdown-pages ( stasis/slurp-directory "resources/md" #"\.md$" ) ) } ) ) ( defn prepare-pages [ pages ] ( zipmap ( keys pages ) ( map # ( fn [ req ] ( highlight-code-blocks % ) ) ( vals pages ) ) ) ) ( defn get-pages [ ] ( prepare-pages ( get-raw-pages ) ) )

Reloading the markdown page should show you that what we've done so far both kinda worked and kinda didn't.

We are getting highlighted code. However, Pygments includes some unwanted wrapping markup. As we're already inside a pre element, the result is not quite as desired. To fix this we will use enlive once more to massage the output from Pygments. The output includes a wrapper div and a pre, let's extract just the code:

( defn- extract-code [ highlighted ] ( -> highlighted java.io.StringReader. enlive/html-resource ( enlive/select [ :pre ] ) first :content ) ) ( defn- highlight [ node ] ( let [ code ( ->> node :content ( apply str ) ) lang ( ->> node :attrs :class keyword ) ] ( assoc node :content ( -> code ( pygments/highlight lang :html ) extract-code ) ) ) )

Enlive's select function selects elements from a document, but unlike sniptest , it does not accept a string. Instead, we must go through its html-resource function, which only accepts input streams. The end result is that we do a select on what Pygments gives us in order to get just the highlighted code. Refreshing the blog post shows that it works as expected.

The -> is the thread-first macro. It works like thread-last, except it threads values as the first argument to the next function. The above example could be written in either of these ways, but most people find the threading form to be the easiest on the eyes:

( -> highlighted java.io.StringReader. enlive/html-resource ( enlive/select [ :pre ] ) first :content ) ( :content ( first ( enlive/select ( enlive/html-resource ( java.io.StringReader. highlighted ) ) [ :pre ] ) ) )

To add some styling, pick a CSS file from the suggested themes repo, and load it onto the page. Update the layout-page in web.clj to look like this:

(defn layout-page [request page] (html5 [:head [:meta {:charset "utf-8"}] [:meta {:name "viewport" :content "width=device-width, initial-scale=1.0"}] [:title "Tech blog"] [:link {:rel "stylesheet" :href "/pygments-css/autumn.css"}]] [:body [:div.logo "cjohansen.no"] [:div.body page]]))

As we're adding more features to our site, it is becoming apparent that processing all the pages to completion on every request isn't ideal. Fixing this is quite easy with Stasis, because we can give Stasis functions instead of strings, and then Stasis will call the function to build a particular page only when it needs to render that specific page.

To make our pages lazy, update prepare-pages to replace the values with functions instead of strings of highlighted HTML. The function should take one argument, the request map.

( defn prepare-pages [ pages ] ( zipmap ( keys pages ) ( map # ( fn [ req ] ( highlight-code-blocks % ) ) ( vals pages ) ) ) )

By having the function literal return a new function that takes one argument, we have significantly improved performance for our development server.

Now that we have a blog with syntax highlighting, we need to start thinking about delivery. Fast webpages beat slow ones on all sorts of metrics. One way to make our site faster is by employing various frontend asset optimization techniques. For this purpose, there is (among others) Optimus. We will use it to:

Concatenate CSS and JavaScript files

Minify CSS and JavaScript files

Serve CSS and JavaScript from cache-friendly URLs

A "cache friendly" URL is one that is unique every time the contents of the URL changes. This way we can serve assets with aggressive cache headers, and users will only need to download them once. The next time we deploy, if the assets have changed, they will have a new URL. Optimus facilitates this by providing some functions to help us link to assets.

We will start by adding Optimus as a dependency in project.clj. Remember to restart the server after doing this.

( defproject cjohansen-no "0.1.0-SNAPSHOT" :description "cjohansen.no source code" :url "http://cjohansen.no" :license { :name "BSD 2 Clause" :url "http://opensource.org/licenses/BSD-2-Clause" } :dependencies [ [ org.clojure/clojure "1.5.1" ] [ stasis "1.0.0" ] [ ring "1.2.1" ] [ hiccup "1.0.5" ] [ me.raynes/cegdown "0.1.1" ] [ enlive "1.1.5" ] [ clygments "0.1.1" ] [ optimus "0.14.2" ] ] :ring { :handler cjohansen-no.web/app } :profiles { :dev { :plugins [ [ lein-ring "0.8.10" ] ] } } )

Now update the web.clj namespace form to require some functions from Optimus:

( ns cjohansen-no.web ( :require [ optimus.assets :as assets ] [ optimus.optimizations :as optimizations ] [ optimus.prime :as optimus ] [ optimus.strategies :refer [ serve-live-assets ] ] [ cjohansen-no.highlight :refer [ highlight-code-blocks ] ] [ clojure.java.io :as io ] [ clojure.string :as str ] [ hiccup.page :refer [ html5 ] ] [ me.raynes.cegdown :as md ] [ stasis.core :as stasis ] ) )

Instead of having Stasis serve the files in public, we will hand them to Optimus as assets. We will define a separate function for these assets, as it makes for a natural place to add further assets and/or bundles of assets later:

( defn get-assets [ ] ( assets/load-assets "public" [ #".*" ] ) )

If your CSS files use @import , Optimus will (by default) take care to inline the import, so there is no need to define bundles at this point. Refer to the Optimus readme for more details.

To make our app use the new assets, we will change the app function:

( def app ( optimus/wrap ( stasis/serve-pages get-pages ) get-assets optimizations/all serve-live-assets ) )

The call to stasis/serve-pages returns a function (a Ring app, remember?) optimus/wrap returns another function with the same signature that wraps the original one. We pass it the function to get all our assets, optimization rules (a function) and a strategy for serving the assets (also a function). optimizations/all is a grab bag of every trick Optimus knows:

Minify JavaScript

Minify CSS

Inline CSS imports

Concatenate bundles

Add cache-bust expires headers (replace URL references with generated unique ones)

Add last-modified headers

You are free to pick and choose from this list if you want, but for most cases, optimizations/all is what you want.

Lastly, we employed Optimus' serve-live-assets strategy, which means that Optimus will read assets from disk on every request. This is useful in development mode, but in a production setting, you would typically use one that's less resource intensive, like serve-frozen-assets .

Create a CSS file and make sure it gets included from the page layout.

import url (../ pygments-css / autumn . css ); body { font : 16 px Helvetica , arial , freesans , clean , sans-serif ; line-height : 1.5 ; margin : 0 10 px ; }

Refreshing the blog in the browser should display the same page as before. However, if you hit the CSS file directly, you will find that Optimus has done what it can to optimize serving it.

There is one final thing to take care of. In production, we can configure our web server to serve assets with a far future expires header. But in order for that to be safe, we need distinct URLs for every change to the file. Let's add another require to the web namespace form:

( ns cjohansen-no.web ( :require [ optimus.assets :as assets ] [ optimus.link :as link ] [ optimus.optimizations :as optimizations ] [ optimus.prime :as optimus ] [ optimus.strategies :refer [ serve-live-assets ] ] [ cjohansen-no.highlight :refer [ highlight-code-blocks ] ] [ clojure.java.io :as io ] [ clojure.string :as str ] [ hiccup.page :refer [ html5 ] ] [ me.raynes.cegdown :as md ] [ stasis.core :as stasis ] ) )

With this in place, we can use Optimus to generate the link to the CSS file. However, to do that, it needs access to the request map, so we need to change a few things. We will start with the layout-page function:

( defn layout-page [ request page ] ( html5 [ :head [ :meta { :charset "utf-8" } ] [ :meta { :name "viewport" :content "width=device-width, initial-scale=1.0" } ] [ :title "Tech blog" ] [ :link { :rel "stylesheet" :href ( link/file-path request "/styles/main.css" ) } ] ] [ :body [ :div.logo "cjohansen.no" ] [ :div.body page ] ] ) )

Both the partial-pages and markdown-pages need to pass the request to layout-page . If we change them to return functions, Stasis will call those functions with the request.

( defn partial-pages [ pages ] ( zipmap ( keys pages ) ( map # ( fn [ req ] ( layout-page req % ) ) ( vals pages ) ) ) )

Remember that #( ) is a function literal, so the mapping function here is a function that returns another function (which takes a request map as its only argument). The markdown generation is similar, but includes the additional step of running the content through cegdown:

( defn markdown-pages [ pages ] ( zipmap ( map # ( str/replace % #"\.md$" "" ) ( keys pages ) ) ( map # ( fn [ req ] ( layout-page req ( md/to-html % pegdown-options ) ) ) ( vals pages ) ) ) )

Previously these maps contained strings, so we need to update their use now that they're functions. We start with a new function:

( defn prepare-page [ page req ] ( -> ( if ( string? page ) page ( page req ) ) highlight-code-blocks ) )

This function takes a page and a request. Because every page will go through this function, some will be strings, and some will be functions. If the page is a string, we leave it untouched, and if it's a function, we call it with the request map and pipe the result through a series of post-processing steps. There's currently only one step, but the threading macro has set us up for easily adding more steps later. The final piece of the puzzle is to update the prepare-pages function:

( defn prepare-pages [ pages ] ( zipmap ( keys pages ) ( map # ( partial prepare-page % ) ( vals pages ) ) ) )

Again, we use the function literal #( ) . We also use partial . This returns a new function that knows the first argument to pass to prepare-page . When you call this new function with one argument (a request), the prepare-page function will be called with a page and a request. Update the page in the browser, view source and note that Optimus has now given our CSS file a nice and unique URL.

So far we've only surfed the server version, but the whole point of this exercise was to create something that can work as a static site. To dump the file to disk, start by adding a custom Leiningen build alias in project.clj:

( defproject cjohansen-no "0.1.0-SNAPSHOT" :description "cjohansen.no source code" :url "http://cjohansen.no" :license { :name "BSD 2 Clause" :url "http://opensource.org/licenses/BSD-2-Clause" } :dependencies [ [ org.clojure/clojure "1.5.1" ] [ stasis "1.0.0" ] [ ring "1.2.1" ] [ hiccup "1.0.5" ] [ me.raynes/cegdown "0.1.1" ] [ enlive "1.1.5" ] [ clygments "0.1.1" ] [ optimus "0.14.2" ] ] :ring { :handler cjohansen-no.web/app } :aliases { "build-site" [ "run" "-m" "cjohansen-no.web/export" ] } :profiles { :dev { :plugins [ [ lein-ring "0.8.10" ] ] } } )

This configures lein build-site as a command that will invoke the export function in the cjohansen-no.web namespace. Stasis gives us what we need to build this function:

( def export-dir "dist" ) ( defn export [ ] ( stasis/empty-directory! export-dir ) ( stasis/export-pages ( get-pages ) export-dir ) )

While this won't technically fail, it also won't be the whole picture. Had we not been using Optimus, this would be OK. Since we are using Optimus, we want to make sure the export is optimized as well. The fix is simple; tell Optimus to dump assets for us, and add an entry to Stasis' request map extensions so that Optimus finds the assets. First update the namespace form to require the Optimus export library:

( ns cjohansen-no.web ( :require [ optimus.assets :as assets ] [ optimus.export ] [ optimus.link :as link ] [ optimus.optimizations :as optimizations ] [ optimus.prime :as optimus ] [ optimus.strategies :refer [ serve-live-assets ] ] [ cjohansen-no.highlight :refer [ highlight-code-blocks ] ] [ clojure.java.io :as io ] [ clojure.string :as str ] [ hiccup.page :refer [ html5 ] ] [ me.raynes.cegdown :as md ] [ stasis.core :as stasis ] ) )

Then update the export function:

( defn export [ ] ( let [ assets ( optimizations/all ( get-assets ) { } ) ] ( stasis/empty-directory! export-dir ) ( optimus.export/save-assets assets export-dir ) ( stasis/export-pages ( get-pages ) export-dir { :optimus-assets assets } ) ) )

Now, on the command line, run lein build-site . After a short while you will find your entire site ready to ship in the dist directory. This can be directly rsynced to your server.

Building sites like we've done in this post opens for various interesting ways of programatically performing tests and health checks. I will show you two simple, yet immensely useful tests we can add to a site of this kind. You can of course also add unit tests for individual functions, and doing so in a system composed of mostly pure functions is very straight-forward, yet outside the scope of this post.

One nice test to put in a site like this is an integration test that checks that every page renders without errors. We will use Midje for our tests, so let's update project.clj:

( defproject cjohansen-no "0.1.0-SNAPSHOT" :description "cjohansen.no source code" :url "http://cjohansen.no" :license { :name "BSD 2 Clause" :url "http://opensource.org/licenses/BSD-2-Clause" } :dependencies [ [ org.clojure/clojure "1.5.1" ] [ stasis "1.0.0" ] [ ring "1.2.1" ] [ hiccup "1.0.5" ] [ me.raynes/cegdown "0.1.1" ] [ enlive "1.1.5" ] [ clygments "0.1.1" ] [ optimus "0.14.2" ] ] :ring { :handler cjohansen-no.web/app } :aliases { "build-site" [ "run" "-m" "cjohansen-no.web/export" ] } :profiles { :dev { :plugins [ [ lein-ring "0.8.10" ] ] } :test { :dependencies [ [ midje "1.6.0" ] ] :plugins [ [ lein-midje "3.1.3" ] ] } } )

We've added a test profile that includes the midje dependencies. Add the following to test/cjohansen_no/web_test.clj:

( ns cjohansen-no.web-test ( :require [ cjohansen-no.web :refer :all ] [ midje.sweet :refer :all ] ) ) ( fact "All pages respond with 200 OK" ( doseq [ url ( keys ( get-pages ) ) ] ( let [ status ( :status ( app { :uri url } ) ) ] [ url status ] => [ url 200 ] ) ) )

We simply call our get-pages function, loop the resulting map, and call each page function with a request map consisting only of a URL. The comparison is made with a vector of the URL and the status. The reason for this is that the URL will be included in the error message if this fails. This way we can know which pages fail. To run the tests:

lein with-profile test midje

Doing this will inform us that the generated core_test.clj fails. Just delete it. Other than that, the test confirms that all is well with our site. To keep the tests running while working on the site, run autotest:

lein with-profile test midje :autotest

Another useful test to have in place is a link-checker. We will make one that at least verifies that the internal links between pages in our app are correct, and that they don't cause any unnecessary redirects (e.g. from /about to /about/).

Enlive is very useful for these things. We will use the select function to find all links, and then make sure that the href attribute points to an existing URL if it is a path (not a full URL, which is treated as an external link). First up is the link-valid? function, which checks if a single link is valid given a map of pages:

( defn link-valid? [ pages link ] ( let [ href ( get-in link [ :attrs :href ] ) ] ( or ( not ( .startsWith href "/" ) ) ( contains? pages href ) ( contains? pages ( str href "index.html" ) ) ) ) )

The link is considered valid if the href attribute either points to a URL that isn't a path within our app (relative paths are assumed not used) or if it points to one of the pages in the map. We're lenient enough to allow links to /about/ when we have /about/index.html. Since we will use enlive to select all the links, update the test namespace form to this:

( ns cjohansen-no.web-test ( :require [ cjohansen-no.web :refer :all ] [ midje.sweet :refer :all ] [ net.cgrand.enlive-html :as enlive ] ) )

Then add the test itself:

( fact "All links are valid" ( let [ pages ( get-pages ) ] ( doseq [ url ( keys ( get-pages ) ) link ( -> ( :body ( app { :uri url } ) ) java.io.StringReader. enlive/html-resource ( enlive/select [ :a ] ) ) ] ( let [ href ( get-in link [ :attrs :href ] ) ] [ url href ( link-valid? pages link ) ] => [ url href true ] ) ) ) )

Again, we loop all the pages and get them. For each page, we select all links, and expect all of them to pass the link checker. Again we make a slightly strange comparison in the interest of having more than true/false in the output if one of these fail. If you add an invalid link to the markdown file now, running the tests will produce this:

FAIL "All links are valid" at (web_test.clj:21) Expected: ["/my-first-post" "/about/" true] Actual: ["/my-first-post" "/about/" false]

18 lines of code to verify all links on the site. Pretty nifty! Rather than having numerous tests that load all the pages, it would probably be a good way to change the structure of the tests such that we only load each page once, and instead register various test functions we want to run for each page. This is left as an exercise for the reader.

I hope this post has shown you the power and flexibility of Stasis and all the other tools. Perhaps it has even convinced you further of the value of Clojure. I really do dislike number-of-lines-of-code jerkoffs, but it is worth mentioning that we were able to build a reasonably feature-complete technical blog in roughly 100 lines of code using a simple, yet powerful "no batteries included" library like Stasis (which itself clocks in at just over 100 lines of code). I hope you will consider Clojure and Stasis for your next semi-static web project.

Big thanks to Magnar Sveen for proof-reading and correcting this post.