For this we're going to tap into a few third party libraries (all available via Quicklisp) to make our life easier.

This is done by either directly typing:

( ql:quickload :library-name-here )

or by adding to the language-popularity.asd file (ASDF - Another System Definition Facility) and including in the :use portion of our defpackage.

Now we should have a language-popularity.asd file that looks like:

:sxql ;; for other tasks :drakma :cl-json :split-sequence :vecto :md5 )

and a defpackage in src/model.lisp that looks like:

( defpackage language-popularity.model ( :use :cl :cl-json :drakma :vecto :md5 :split-sequence ) ( :import-from :language-popularity.config :config ) ( :export :get-language-sub-stats :pie-chart ) )

Ignore the #'pie-chart and :vecto portions will come later, so lets just add a stub/placeholder function for that for now (in src/model.lisp), as well as a very basic class structure for our Language type class (classes in Common Lisp are done via a system called CLOS, which is very awesome, but we're mainly using it as an easy way to store and reference values):

( defclass Language () ( ( Subscribers :accessor subs :initarg :subs :initform 0 ) ( Last -Updated :accessor last-updated :initarg :last-updated :initform 0 ) ( About :accessor about :initarg :about :initform "Some details about the language." ) ) ( :documentation "Language stats and details" ) ) ( defun pie-chart ( slices ) slices )

Next up, we need some functions to:

Query a remote JSON endpoint (did you know you can add a .json extension to most any reddit page and get a JSON response? pretty sweet!)

Decode the remote JSON into a string for cl-json to parse

Parse the JSON string with cl-json to get an object we can use

Add this next (still in src/model.lisp unless I say otherwise!):

( defun char-vector-to-string ( v ) ( format nil "~{~a~}" ( mapcar #'code-char ( coerce v 'list ) ) ) ) ( defun remote-json-request ( uri ) "Pull in remote JSON. Drakma returns it as a large vector of character codes, so we have to parse it out to string form for cl-json." ( let* ( ( json-response-raw ( http-request uri ) ) ( json-response-string ( char-vector-to-string json-response-raw ) ) ( json ( decode-json-from-string json-response-string ) ) ) json )

The #'http-request function is courtesy of drakma (similar to CURL in other languages), while #'decode-json-from-string is from cl-json.

Also, for the Common Lisp novices, #' is a way to refer to a function, so I'll typically add it in front to distinguish it as such.

The drakma #'http-request gives a vector set of ASCII character codes, so our little #'char-vector-to-string changes it to a basic JSON string.

Now that we have a way to remotely request some parsed out JSON, we can tap into the reddit endpoints and store some instances of our new Language objects with actual data.

For this, we'll just throw them in a hash table, as it'll give us an easy way to query out details based on a language name.

Add this:

( defparameter *language-stats* ( make-hash-table :test #'equal ) ) ( defconstant +cache-time+ ( * 60 60 ) ) ;; 1 hour ( defmacro jkey ( k &rest rest ) ` ( cdr ( assoc ,k ,@rest ) ) ) ( defun set-language-stats ( language ) "Build language stats into our lang class via external sources of popularity." ( let ( ( lang-class ( or ( gethash language *language-stats* ) ( make-instance 'Language ) ) ) ) ( when ( > ( - ( get-universal-time ) ( last-updated lang-class ) ) +cache-time+ ) ( let ( ( reddit-json ( remote-json-request ( format nil "http://reddit.com/r/~a/about.json" language ) ) ) ) ( when ( jkey :subscribers ( jkey :data reddit-json ) ) ( setf ( subs lang-class ) ( jkey :subscribers ( jkey :data reddit-json ) ) ) ) ( setf ( last-updated lang-class ) ( get-universal-time ) ) ) ) ( setf ( gethash language *language-stats* ) lang-class ) ( cons ( intern ( string-upcase language ) ) ( subs lang-class ) ) ) )

Holy freaking function Batman! May be your first reaction to seeing this abomination of a function; I assure you it's not so bad, although sometimes due to the interactive nature of the Common Lisp REPL and adding little pieces at a time, it is easy to get carried away and not properly modularize enough of the code.

So, lets take it line by line.

The language-stats and cache-time parameter and constants are there for storing Language objects in the former, and keeping our API requests to reddit down in the latter.

The macro #'jkey (ehh…not a function, but I'll steal the #' syntax) is to reduce verbosity in the #'set-language-stats by a small amount.

It lets us quickly reference nested keys in a JSON alist created by cl-json.

Now we enter the beast (I mean #'set-language-stats). It takes a single parameter/argument (language) and runs a JSON request to http://reddit.com/r/language/about.json, as long as it's been at least an hour since our last request (restarting the instance of your implementation is obviously going to reset this).

When the subscriber key exists (if you request an invalid endpoint, it won't), it will update the Language object's Subscribers slot.

Following the subscriber update, it adds (or updates) the language's value under it's name key in the language-stats parameter (yea, yea, global state, mutability, not ideal, I get it).

Finally, after all of that, we have a little laziness tossed in. The function ends up returning a cons of the language name interned (so "haskell" becomes HASKELL) and the subscriber count.