Using Google Data API’s with Clojure

October 27, 2010 at 11:38 am | Posted in Clojure, Programming | 8 Comments



With the Google Data API it is very easy to access all that information that you have trusted to ‘don’t be evil’ Google. The basic format lends itself very well to manipulating with Clojure, but in this blog I’ll just show how to access your contacts data, building on top of a Java client library.

I used Leiningen for this experiment. To start I created a new project with

$ lein new gdata

This will create a new directory structure containing our project. First we need to add some dependencies to project.clj:

(defproject gdata "1.0.0-SNAPSHOT" :description "Google Data API experiment" :dependencies [[org.clojure/clojure "1.2.0"] [org.clojure/clojure-contrib "1.2.0"] [com.google.gdata/gdata-contacts-3.0 "1.41.5"]] :repositories {"mandubian-mvn" "http://mandubian-mvn.googlecode.com/svn/trunk/mandubian-mvn/repository"})

I have added an extra dependency on line 5 because we need the Contacts Java client library. Since it isn’t included in the standard Maven repositories I also had to add an extra one. This is shown in line 6. Someone was kind enough to Mavenize all the Google Data client jars.

Make sure you check this setup by executing:

$ lein deps

I will explain the code (src/gdata/core.clj) here as separate blocks. At the end of this blog you will see the complete listing. First I start with defining the namespace and a couple of imports:

(ns gdata.core) (import '(com.google.gdata.client.contacts ContactsService) '(com.google.gdata.data.contacts ContactFeed) '(java.net URL))

Both ContactsService and ContactFeed are classes in the Java Contacts client library.

The next block is optional and only needed if you are working behind a firewall.

(System/setProperty "https.proxyHost" "your-proxy-name-or-ip") (System/setProperty "https.proxyPort" "8080") (System/setProperty "http.proxyHost" "your-proxy-name-or-ip") (System/setProperty "http.proxyPort" "8080")

The proxy port is usually 8080 but could be different. Please double check.

Next we instantiate the Contacts service:

(defn set-user-credentials [cs username password] (.setUserCredentials cs username password)) (defn get-service-version [cs] (.getServiceVersion cs)) (defn contact-service [service-name username password] (let [cs (new ContactsService service-name)] (set-user-credentials cs username password) cs))

The set-user-credentials function is not really needed here and could also be in-lined. The get-service-version is just a wrapper around the Java member function.

Next we define a couple of functions that allow us to retrieve a stream of contacts:

(defn get-feed [cs url] (.getFeed cs url (class (new ContactFeed)))) (defn fetch-url [username max-results] (str "http://www.google.com/m8/feeds/contacts/" username "/full?max-results=" max-results)) (defn get-entries [feed] (.getEntries feed))

The function get-feed returns a ContactFeed object. The getFeed Java function has the class of the feed as parameter. I tried to pass ContactFeed/class but that didn’t work. The workaround is to instantiate a new ContactFeed object, and then ask it’s class. The next function is fetch-url. This function creates the url that is used to retrieve the feed. It hides the leaky abstraction that the class ContactsService somehow doesn’t. Finally get-entries returns a Java List of ContactEntry.

Now that all building blocks are in place we can define a couple of functions that retrieve some actual contact data, like e-mail addresses:

(defn get-email-addresses [entry] (.getEmailAddresses entry)) (defn get-address [email] (.getAddress email)) (defn get-first-email-address [entry] (let [email-addresses (get-email-addresses entry)] (if (empty? email-addresses) "no address" (get-address (first email-addresses)))))

get-email-addresses is a wrapper function. It returns a list of Email objects. This list can be empty. Given an Email object, the function get-address returns a string representation of the email address. The last funtion, get-first-email-address checks if the contact has at least one email address. If not, it returns “no address”, otherwise it returns the very first address in the list.

Finally the complete program:

(ns gdata.core) (import '(com.google.gdata.client.contacts ContactsService) '(com.google.gdata.data.contacts ContactFeed) '(java.net URL)) ;(System/setProperty "https.proxyHost" "your-proxy-name-or-ip") ;(System/setProperty "https.proxyPort" "8080") ;(System/setProperty "http.proxyHost" "your-proxy-name-or-ip") ;(System/setProperty "http.proxyPort" "8080") (defn set-user-credentials [cs username password] (.setUserCredentials cs username password)) (defn contacts-service [service-name username password] (let [cs (new ContactsService service-name)] (set-user-credentials cs username password) cs)) (defn get-service-version [cs] (.getServiceVersion cs)) (defn get-feed [cs url] (.getFeed cs url (class (new ContactFeed)))) (defn fetch-url [username max-results] (str "http://www.google.com/m8/feeds/contacts/" username "/full?max-results=" max-results)) (defn get-entries [feed] (.getEntries feed)) (defn get-email-addresses [entry] (.getEmailAddresses entry)) (defn get-address [email] (.getAddress email)) (defn get-first-email-address [entry] (let [email-addresses (get-email-addresses entry)] (if (empty? email-addresses) "no address" (get-address (first email-addresses))))) (defn main [] (let [username "your-username@gmail.com" password "your-gmail-password" cs (contacts-service "clojure-test" username password) url (new URL (fetch-url username 100)) feed (get-feed cs url) entries (get-entries feed)] (println "version: " (get-service-version cs)) (println "count: " (count entries)) (println (map #(get-first-email-address %) entries))))

Using other Google Data client API’s with Clojure is just as straightforward. There are a couple of things that could be improved:

the Java client library has some leaky abstractions: why do I as a user of this library have to know about url’s? Why do I have to know how many data (in the above sample 100) I want to retrieve? We could hide these problems in our Clojure wrapper by creating a lazy sequence that retrieves the next set of data on an as-needed basis. we could build a Clojure client instead of using the Java client. This would mean that we have to build something directly on top of the Google Data protocol. Clojure as a language is perfectly suited for parsing these kind of data streams.

Hopefully this blog will enable you to write some cool Clojure applications on top of Google’s Data API.