Introduction

At Workshape.io, we're always looking into ways to improve our analytics tool for users. Now that we have have over 8,000 users on our platform we wanted to create an interactive tool that would enable developers and employers to dig a little deeper into our dataset to find out more about the supply and demand of developers.

This blog post is a walkthrough of how we created an interactive choropleth map to power our new analytics interface.

Inspiration

One of the catalysts for bringing our new analytics to life was a tutorial written by Mike Bostock about creating a map using d3.js and topojson. (Anyone new to map making, I strongly advise checking out this tutorial to demystify and simplify the process!) After completing the tutorial myself, I became familar with 3 of the main elements that we used to create our interactive map:

Natural Earth - a data source for geographic data

Geospatial Data Abstraction Library - a command line tool for manipulating geographic data, that can output GeoJSON

Topojson - an extension of GEOJson that optimises the underlying data structure, removing redundant encoding of boundaries to reduce file sizes.

The final ingredient and the main inspiration came from another D3.js pioneer, Jason Davies, with his website providing examples using orthographic projections (mapping the world onto a sphere in two-dimensional space).

Steps

1. Creating topojson

In order to draw a map we first need to get the underlying geographic data for doing so. NaturalEarth is a one-stop shop for free vector and raster map data which provides the data we need. It is a comprehensive source of geographic data that vastly simplifies the process of creating a map.

For our component we are going to use 1:110m scale cultural data and download two sets of data:

This data will allow us to have shapes for all countries, except the US, and US states. Once we have downloaded the data and extracted the data from the files we are going to use ogr2ogr and topojson to create of topojson files.

N.B If you have not installed these on your machine, please refer to Mike Bostock's blog post.

Our usage of the command is quite simple. We specify a format we want to generate, an SQL-like WHERE clause to ignore the shape for the USA, an output file name and an input .shp file.

ogr2ogr -f GeoJSON -where 'iso_a3 <> "USA"' countries.json ne_110m_admin_0_countries/ne_110m_admin_0_countries.shp

This will generate a GeoJSON file containing features for each country encoded in the .shp file. This file is 677kb.

The command for converting the state .shp file into GeoJSON is a bit simpler, as we do not need to limit the output with a WHERE clause.

ogr2ogr -f GeoJSON states.json ne_110m_admin_1_states_provinces/ne_110m_admin_1_states_provinces.shp

This generates a 145kb file. We need to massage the data slightly so that we can have a consistently named column with similar valued data. The following command replaces values in place in states.json so that we have a property called iso_n3.

sed -i '' 's/"adm1_code": "USA-/"iso_n3":"/' states.json

At this stage we now have 2 GeoJSON files. We could decide to use these files to generate a choropleth map, however the combined size of 822kb. One of the main benefits of Topojson over GeoJSON is that it eliminates redundancy (e.g. by not specifiying coordinates of boundaries more than once). This means we can use topojson to shrink our combined file size, whilst also concatenating the files together. We can reduce the number of connections to our server and also reduce the download size, creating a better user experience.

We run the following command, which specifies the output file, the property to set as id, any properties to keep and then all input files to process.

topojson -o countries-and-states.json --id-property iso_n3 --properties name=name -- countries.json states.json

What we end up with is one file size 104kb, approximately a 85% reduction in size - a lot better for loading in the browser. We'll use this in conjunction with topojson, which is 15kb uncompressed, to convert back into GeoJSON ready for rendering with D3.js.

2. Creating a Directive

Now that we have created our own topojson file, we need to create a Angular directive, that we can use in our app for creating a choropleth. The four main ingredients to this are:

d3.js

topojson.js

countries-and-states.json

a set of data containing values for each feature

We'll focus on first building an interactive spherical projection of the world, then we'll get to the data.

To encapsulate all the functionality of the visualisation we'll use an Angular directive. This way we'll enable anyone building the app to access the functionality with a simple API e.g. <globe data='somedata'></globe> . We'll create a directive that allows for the data to be passed in, this will be what is plotted onto the choropleth. The directive will then handle getting the topojson file and drawing the map.

app.directive("globe", function() { return { restrict : 'E', scope : { data: '=?' }, template: '<div class="globe-wrapper">' + '<div class="globe"><!-- D3 Globe will go here --></div>' + '<div class="info"><!-- Text based information will go here --></div>' + '</div>', link: function() {} }; });

The most complex part of building this component is understanding D3.js and its geo API and how projections work. If you take a look at the Geo Projections documentation it gives a clear and thorough introduction to how they work. This should hopefully de-mystify how to achieve this! We'll be using an orthographic projection to create a 3d representation of Earth in 2d space! To acheive this we can use the following snippet.

var projection = d3.geo.orthographic() .translate([width / 2, height / 2]) .scale(250) .clipAngle(90) .precision(0.1) .rotate([0, -30]); var path = d3.geo.path().projection(projection); var svg = d3.select(element[0]).select('.globe') .append('svg') .attr('width', width) .attr('height', height) .attr('viewBox', '0, 0, ' + width + ', ' + height); var features = svg.append('g'); features.append('path') .datum({type: 'Sphere'}) .attr('class', 'background') .attr('d', path);

This creates an orthographic projection; translates the coordinates 0', 0' to be in the center of the allocated space; sets the scale; sets the clipping angle to 90 degrees, so that we see a globe; provides a precision value for adaptive resampling; and then finally sets the default orientation of the sphere, we'll be centred on Algeria!

We then use d3.geo.path() to create an SVG generator from our projection, create an SVG element with a width and height, and then draw a Sphere. You can see this in (version 1).

So that's great but all we have is a here sphere here. What will help understand the projection better is to draw a graticule showing meridians and parallels. D3's geo API has a handy function for generating this, so it is quite trivial.

var graticule = d3.geo.graticule(); features.append('path') .datum(graticule) .attr('class', 'graticule') .attr('d', path);

Cool, so now we can see (version 2) how the sphere is an orthographic projection with the graticule signifying how we've projected each point into 2d space.

No we can move onto plotting the topojson data using this projection. We'll use D3's function for asynchronously loading some json and then use topojson to extract GeoJSON features from the topojson file.

d3.json(mapJson, function(error, world) { states = topojson.feature(world, world.objects.states).features; countries = topojson.feature(world, world.objects.countries).features; stateSet = drawFeatureSet('state', states); countrySet = drawFeatureSet('country', countries); }); function drawFeatureSet(className, featureSet) { var set = features.selectAll('.' + className) .data(featureSet) .enter() .append('g') .attr('class', className) .attr('data-name', function(d) { return d.properties.name; }) .attr('data-id', function(d) { return d.id; }); set.append('path') .attr('class', 'land') .attr('d', path); set.append('path') .attr('class', 'overlay') .attr('d', path) .attr('style', function(d) { if (scope.data[d.id]) { return 'fill-opacity: ' + (scope.data[d.id]/100); } }) .on('click', function(d) { var val = (scope.data[d.id]) ? scope.data[d.id] : 0; d3.select(element[0]).select('.info').html(d.properties.name + ': ' + val); }); return set; }

In the above example we've also written a function to take a set of features and draw it onto sphere. If we now take a look (version 3) at this in the browser, we can see the countries and states drawn onto the sphere.

We now have a map! But what about if we want to see Australia? What we need to do is add some interactivity to the globe, so like in real-life, we can spin it around to see countries on the other side of the world. Luckily for us, Jason Davies, a data visualisation specialist, has written an extension to d3's geo API that encapsulates the behaviour we wish to add.

If we include the javascript file containing the extension, then create our zoom behaviour and invoke it upon all path elements once our data is loaded then the globe will be interactive.

zoom = d3.geo.zoom() .projection(projection) .scaleExtent([projection.scale() * 0.7, projection.scale() * 8]) .on('zoom.redraw', function(){ d3.event.sourceEvent.preventDefault(); svg.selectAll('path').attr('d',path); }); d3.json(mapJson, function(error, world) { states = topojson.feature(world, world.objects.states).features; countries = topojson.feature(world, world.objects.countries).features; stateSet = drawFeatureSet('state', states); countrySet = drawFeatureSet('country', countries); d3.selectAll('path').call(zoom); });

We specify the projection we are using, the minimum and maximum scale the user can be zoom in or out to and then finally the redraw handler and we have added interaction controls to our map. The user can now spin and zoom in on the globe (version 4).

A final touch you may want to add to the map before adding data, is auto-focussing on a country once clicked. We can write a function that, given a feature, find it's centroid, then creates a transition, that lasts 1.25 seconds spinning the globe from it's current focus to the centre of the clicked country.

function rotateToFocusOn(x) { var coords = d3.geo.centroid(x); coords[0] = -coords[0]; coords[1] = -coords[1]; d3.transition() .duration(1250) .tween('rotate', function() { var r = d3.interpolate(projection.rotate(), coords); return function(t) { projection.rotate(r(t)); svg.selectAll('path').attr('d', path); }; }) .transition(); }

If we put this function into our direction and then call it in side our click event handler, we now have auto-focussing too (version 5).

3. Adding Data onto the Map

The final ingredient can be as simple as an hash map, with keys equal to an iso_n3 and an associated value (for simplicity's sake a value between 0 and 100). We can use this to set the colour to render each country polygon. By creating an object in our controller and passing it into our globe directive we now see a choropleth (version 6)!

Wrap Up

Creating your own map can be a daunting task. But with tools at your disposal like naturalearthdata.com, GDAL, topojson and d3.js, it is a lot more approachable. You can quickly put together a compelling, and interactive, geographic visual combining this tutorial with some of your own data.

Here's a screenshot of our effort - Developer Stats, giving a territory-by-territory breakdown of the developers currently on our site. You can access this data through the employer side of the service.

Thanks have to go to Mike Bostock and Jason Davies for the pioneering work they have done that make this task relatively pain-free!

I hope you found this walkthrough useful and can put the directive to use in your own projects.