In this blog post I’ll talk about how I built a visualization of Toronto Transit vehicle locations and some of the engineering challenges I ran into. A working demo can be found on my personal website (I’ve noticed some corporate networks blocking port 3000 and this application requires it), and all code can be found on github.

Picking Technologies

1. Tech Stack

The first step in building this app was to choose a tech stack to work with. I wanted a stack that would:

Let me view on as many platforms as possible

Not have to deal with an App Store publish / review process

I decided to build a web app using React.

2. Map Library

There were a couple of different things I wanted from a mapping library:

Be really fast

Work well with react, or be easy to wrap inside a react component

Be customizable, in order to prevent it from appearing just like google maps

React-Map-GL, a library built by Uber seemed to be a great candidate. React-Map-GL is built on top of Mapbox-Gl-Js, which in turn works using WebGL.

3. Data Source

Most importantly, I had to find an API that gives me the data I want. I found an API called Nextbus, which provides this data in XML. I then found a wrapper around it called Restbus that provides the same data as JSON. Restbus is provided as an easy to use node library as well as a public url.

4. Hosting infrastructure

I decided to use an AWS EC2 instance to host the project.

Building the app

Backend

Getting the backend to work seemed really straightforward at first. I could just make requests to the restbus API at http://restbus.info/api/agencies and display it on a map. However, I ran into some problems with this approach:

Problem #1: The Restbus API is hosted on HTTP, while my website is hosted on HTTPS. Most browsers block HTTPS websites from requesting insecure resources.

Problem #2: I noticed that the public Restbus API went down when I made too many consecutive requests. This could be due to some sort of rate limiting.

I realized I could overcome Problem #1 by deploying my own instance of the Restbus API that calls Nextbus. I could then solve any rate limits imposed by Nextbus by caching responses for a few seconds using redis.

This is what my server.js file looked like:

backend server.js file

To use redis, I simply used the docker image provided by them.

Frontend

I used create-react-app to bootstrap the react frontend, and added the react-map-gl library. I ran into some issues with react-map-gl not building correctly (https://github.com/uber/react-map-gl/issues/176), but fixed it by ejecting from create-react-app, and modifying the webpack config to also process files from react-map-gl.

I used RxJS to take care of API requests. I could then subscribe to these streams in my map component. I set it up to make requests every 2 seconds.

This is what my Map.js component first looked like. Locations are obtained from the $vehicles stream and stored in local component state.

Performance Engineering

One of the big issues I ran into was lag. Lots of it while panning the map around. Running the chrome profiler showed me the following:

The component update was taking 359ms to update after each API request. I soon noticed that this is caused due to the fact that all ~900 markers were updating on every request, regardless of what was displayed within the window.

Adding a condition to only update markers that are within the current window boundaries and re-running the chrome profiler showed me the following:

47.27ms instead of 359.84ms for the same zoom level, an improvement of ~750%!

2. I also noticed that network requests take rather long.

Let’s see if the restbus API returns any data we do not need.

We’re only using a few fields out of this — id, routeId, secsSinceLastReport, lat, lon , and everything else can be ignored. Filtering this on the backend using an express middleware before sending a json response gives us:

Response times are on average a lot lower!

3. Functional Components

The React docs recommend writing components as pure stateless functions whenever possible. Future improvements to react will have performance optimizations to components declared this way by avoiding unnecessary checks, memory allocations and lifecycle methods.

A hack to make functional components faster right now is to call them as functions, as opposed to JSX nodes.

const Component = (props) => (

<div>{props}</div>

); // Using component in JSX => slower

<div>

<Component />

</div> // Calling component as a regular function in curly braces => faster

<div>

{Component()}

</div>

Although I haven’t profiled it yet, there are several blog posts that report a big speedup. Here’s one:

Further Reading: