When and where is the next train going?! The IoL City citizens have things to do and places to be!

Goal

Pull in Transport For London API data to get schedule information for a local train stop

data to get schedule information for a local train stop Display next trains and time to arrival on an OLED screen

Technologies & Components

Overview

The OLED display project will build on the skills and technology learned in the Weather Station project. The basic concept is to utilize a Cactus Micro micro controller (Arduino clone with ESP8266 WiFi built-in) with MQTT messaging to exchange information. But instead of the device sending data, it will be receiving data to display on an OLED screen. In addition, the schedule data will be published to a PubNub channel for use with other IoL City projects.

Transport for London API

As a fun exercise, I wanted to pull in some publicly available data via an API, manipulate it and use it with an application. The TFL API was a great fit for this project. It works by simply going to a website, which responds with information in a JSON format. No need for API keys, just a simple GET request and you have data!

So how does it really work? I’m glad you asked. Basically, there are a number of URLs that you can visit that return related information. Anything from trains, buses to boats can be retrieved with their “Unified RESTful API”.

All the information can be found here: https://api-portal.tfl.gov.uk/docs

To keep things simple,I’ve selected a local train station near me which has two outbound routes. I will then use that information to determine which direction the train track should be switched to and also display the time it will arrive.

To find the station ID, I first used the following GET URL



https://api.tfl.gov.uk/StopPoint/Search/fields



Which then returns the following result snippet

{"$type":"Tfl.Api.Presentation.Entities.SearchResponse, Tfl.Api.Presentation.Entities","query":"fields","total":50,"matches":[{"$type":"Tfl.Api.Presentation.Entities.MatchedStop, Tfl.Api.Presentation.Entities","stationId":"940GZZLUNFD","icsId":"1000159","topMostParentId":"940GZZLUNFD","modes":["bus","tube"],"status":true,"id":"940GZZLUNFD","name":"Northfields Underground Station","lat":51.499324,"lon":-0.31472},{"$type":"Tfl.Api.Presentation.Entities.MatchedStop, Tfl.Api.Presentation.Entities","stationId":"940GZZLUSFS","icsId":"1000209","topMostParentId":"940GZZLUSFS","modes":["bus","tube"],"status":true,"id":"940GZZLUSFS","name":"Southfields Underground Station","lat":51.445076,"lon":-0.2066},{"$type":"Tfl.Api.Presentation.Entities.MatchedStop, Tfl.Api.Presentation.Entities","stationId":"910GLONFLDS","icsId":"1001183","topMostParentId":"910GLONFLDS","modes":["overground","national-rail"],"status":true,"id":"910GLONFLDS","name":"London Fields Rail Station","lat":51.541158,"lon":-0.057747}, …...

I then searched for “London Fields” and found that the ID is 910GLONFLDS. Finally, I used the following URL to retrieve the local train schedule.

URL: https://api.tfl.gov.uk/StopPoint/910GLONFLDS/arrivals Which then returns the following result snippet

[{"$type":"Tfl.Api.Presentation.Entities.Prediction, Tfl.Api.Presentation.Entities","id":"-407419356","operationType":1,"vehicleId":"6uXFyoXFIOb2cMfIGrmiLQ==","naptanId":"910GLONFLDS","stationName":"London Fields Rail Station","lineId":"london-overground","lineName":"London Overground","platformName":"2","direction":"outbound","destinationNaptanId":"910GENFLDTN","destinationName":"Enfield Town Rail Station","timestamp":"2016-01-15T18:38:32.787Z","timeToStation":1680,"currentLocation":"","towards":"","expectedArrival":"2016-01-15T19:06:32.787Z","timeToLive":"2016-01-15T19:06:32.787Z","modeName":"overground"},{"$type":"Tfl.Api.Presentation.Entities.Prediction, Tfl.Api.Presentation.Entities","id":"-422333055","operationType":1,"vehicleId":"Br9J3rKECr9QYjK0MhQCaw==","naptanId":"910GLONFLDS","stationName":"London Fields Rail Station","lineId":"london-overground","lineName":"London Overground","platformName":"1","direction":"inbound","destinationNaptanId":"910GLIVST","destinationName":"London Liverpool Street Rail Station","timestamp":"2016-01-15T18:38:32.802Z","timeToStation":2220,"currentLocation":"","towards":"","expectedArrival":"2016-01-15T19:15:32.802Z","timeToLive":"2016-01-15T19:15:32.802Z","modeName":"overground"},{"$type":"Tfl.Api.Presentation.Entities.Prediction, Tfl.Api.Presentation.Entities","id":"1667012363","operationType":1,"vehicleId":"Dk12CzBKtYQmV+mYwrJh6w==","naptanId":"910GLONFLDS","stationName":"London Fields Rail Station","lineId":"london-overground","lineName":"London ….

Cool! But that is way too much info for my purposes. I also want to automate this task and send the results to my Internet of Things LEGO

Programming

I will be using Node-RED to perform the plumbing for my two messaging systems, MQTT & PubNub. Although I wrote much of this application originally in pure NodeJS, I found that Node-RED was still more fun to experiment with. For example, I could quickly pipe this data to Twitter/Slack/Twilio to send me a message if my train is close. By just passing around objects and manipulating them with JS in function nodes, I can pretty much do what I want for IoT related tasks and rapid prototyping.

Node-RED Flow – Schedule and Display

Download flow: Gist

To get the TFL data into the system, I first started by creating a function to define the URL that will be passed to the HTTP Get request node. This was accomplished by using a function node to simply write the msg.url property to the desired station ID. I use an injection node with a repeat schedule to automate the data stream. Because it pulls every second, I placed a delay of 1 minute to not abuse the network. I also added a RESTful interface myself for future use and flexibility.

The API data is returned as a JSON string, which I then use a JSON node to convert it to an object. This is necessary so I can iterate through it and extract the important data.

With the data in hand, its time to extract it and create a new schema. Here’s an example of what a populated data set will look like.

{ "station": "London Fields Rail Station", "schedule": [ { "expected": "3", "destination": "Enfield Town Rail Station" }, { "expected": "15", "destination": "Cheshunt Rail Station" }, { "expected": "25", "destination": "Enfield Town Rail Station" }, { "expected": "35", "destination": "Cheshunt Rail Station" }, { "expected": "50", "destination": "Enfield Town Rail Station" } ] }

But first, the data is originally sent in some arbitrary order, so it needs to be sorted. At the same time, the desired properties are saved to the new schema. This function node takes care of all that:

var tfl = msg.payload; var trains = {}; var station; var expected; var nextTrains = []; // check for valid data first if (msg.statusCode == 200) { //if(tfl[0] !== undefined){ if(tfl[0]){ station = tfl[0].stationName; // sort schedule tfl.sort(function(a,b){ return (a.timeToStation - b.timeToStation); }); // iterate through all schedule records and save required data for (var i = 0; i < tfl.length; i++) { if(tfl[i].direction == 'outbound'){ nextTrains.push({ 'expected': (tfl[i].timeToStation)/60, 'destination': tfl[i].destinationName }); } } }else{ // no schedule received station = msg.stationid; // use station id as default nextTrains.push({ 'expected': 0, 'destination': "Out of Service" }) } trains.station = station; trains.schedule = nextTrains; } // display and send schedule data console.log("outbound schedule new schema > "); console.log(util.inspect(trains, false, null)); msg.payload = trains; flow.set('trainschedule', msg.payload); // save for flow use return msg;

HTTP response node

By adding an HTTP input and response node, I can toggle the train schedule routine and get a current formatted schedule. Although I am not relying on this for this project, I thought it was a handy option for the future. Maybe a UI dashboard can request that data as needed.

GET Request: http://myNodeRedServer:1880/trainschedule

Example of GET response

{"station":"London Fields Rail Station","schedule":[{"expected”:"0","destination":"Cheshunt Rail Station"},{"expected":"18","destination":"Enfield Town Rail Station"},{"expected":"33","destination":"Cheshunt Rail Station"},{"expected":"48","destination":"Enfield Town Rail Station"},{"expected":"63","destination":"Cheshunt Rail Station"}]}

Now that I have the data… the magic begins!

The msg object, which now contains the schedule information, will be passed to multiple nodes for future applications.

MQTT

Send the schedule to the OLED display board.

Since this data is in JSON format, it must first be converted to a simple text string. I also pre-formatted it to avoid too much code on the embedded side. This approach also allows the OLED display board to be easily used for other applications without much adjustment. The schedule was too much text, so the destination names were simplified. Instead of “Enfield Town Rail Station”, the string was split and then the first two words and a space were joined back. I also padded the expected time with a leading 0 if less than 10 so that the station names would line up. Finally, the data was sent to the MQTT topic /trainschedule/910GLONFLDS

Function: Format to MQTT display string

var station = msg.payload.station; var schedule = msg.payload.schedule; var destination; var split; msg.topic = "/trainschedule/" + msg.stationid; if (schedule[0].destination == "Out of Service"){ destination = schedule[0].destination; //destination = "Out of Service"; msg.payload = destination; return msg; }else{ // Split the string and only use the first two words of each value split = station.split(" "); station = split[0] +" "+ split[1] + "



"; msg.payload = station; for(var x = 0; x < schedule.length; x++){ split = schedule[x].destination.split(" "); destination = split[0] +" "+ split[1]; msg.payload += pad(schedule[x].expected) + "m" + " " + destination + "

"; } return msg; } function pad(n) { return (n < 10) ? ("0" + n) : n; }

Schedule Display

Cactus Micro Rev2

An Arduino LillyPadUSB clone with built-in ESP8266 module.

Resources:

http://wiki.aprbrother.com/wiki/Cactus_Micro_Rev2

OLED 128×64

Specifications: 0.96” I2C IIC 128X64 OLED LCD LED

Resources: https://www.adafruit.com/products/326

Text Only Library: https://github.com/greiman/SSD1306Ascii

Circuit

The circuit is very simple. Just connect the OLED board to the VCC and ground pins. Then connect the SDA (GPIO 2) and SDC (GPIO 3) pins.

Note: This Fritzing diagram uses an Arduino Nano, so pin placement will be slightly different.

Code

Before the project code can be uploaded to the Cactus Micro, we must first flash the onboard ESP8266 with the appropriate firmware. To accomplish this, a utility sketch will need to be first uploaded to the Cactus, which will create a transparent bridge to the ESP8266.

Step 1: Configure the Arduino as an ESP programmer

Connect the Cactus Micro to your computer, and set the Arduino IDE to use the LillyPad USB for the board. Then upload the following sketch.

sketch esp8266Programmer

More info:

http://wiki.aprbrother.com/wiki/How_to_made_Cactus_Micro_R2_as_ESP8266_programmer

Step 2: Flash the ESP8266 with this firmware

The ESP8266 is now directly accessible from a serial port and can be programmed with the esp tool. Download the firmware and tool, then run the programming command. Be sure to adjust the serial port (e.g.ty.usbmodem1421) to match your configuration.

Firmware: espduino

Tool: esptool

./esptool.py -p tty.usbmodem1421 write_flash 0x00000 esp8266/release/0x00000.bin 0x40000 esp8266/release/0x40000.bin

Step 3: Program project code to Cactus Micro

The general code flow begins with initializing a timer, OLED screen, ESP8266 WiFi and MQTT messaging.

Since this microcontroller only has 32k of memory, space is a real challenge. I originally tried to use the nice Adafruit graphics library, but it wouldn’t fit on the device with the additional communication libraries. So instead, a text only custom library will be used. I’ve also used the MQTT library provided by April Brothers, who manufacture the Cactus Micro.

OLED Graphics Libary

MQTT Library

Once everything is initialized and the device has connected to the MQTT broker over WiFi, the system waits for an incoming train schedule message. Every few seconds, the screen is updated with the new schedule data and periodically splash a brand page “IoL City Train Network”.

/*** Internet of Lego - Train Schedule Display * This program connects to WiFi, utilizes MQTT and displays the local train schedule on an OLED screen * The OLED graphics library is slimmed down to only text for memory efficiency * https://github.com/greiman/SSD1306Ascii * * Hardware: Cactus Micro Rev2 - http://wiki.aprbrother.com/wiki/Cactus_Micro_Rev2 * Written by: Cory Guynn with some snippets from public examples * 2016 */ // Global Variables unsigned long time; // used to limit publish frequency // OLED #include "SSD1306Ascii.h" #include "SSD1306AsciiAvrI2c.h" // 0X3C+SA0 - 0x3C or 0x3D #define I2C_ADDRESS 0x3C SSD1306AsciiAvrI2c oled; // ESP8266 WiFi #include #define PIN_ENABLE_ESP 13 #define SSID "yourSSID" #define PASS "yourpassword" // MQTT Messaging #include ESP esp(&Serial1, &Serial, PIN_ENABLE_ESP); MQTT mqtt(&esp); boolean wifiConnected = false; #define mqttBroker "aws.internetoflego.com" #define mqttSub "/trainschedule/910GLONFLDS" String mqttMessage = ""; /******************* // Functions *******************/ void wifiCb(void* response) { uint32_t status; RESPONSE res(response); if(res.getArgc() == 1) { res.popArgs((uint8_t*)&status, 4); if(status == STATION_GOT_IP) { wifiConnected = true; Serial.println("WIFI CONNECTED"); oled.print("WiFi Online: "); oled.println(SSID); mqtt.connect(mqttBroker, 1883, false); //or mqtt.connect("host", 1883); /*without security ssl*/ } else { wifiConnected = false; mqtt.disconnect(); oled.println("WiFi OFFLINE"); } } } void mqttConnected(void* response) { Serial.println("MQTT Connected"); oled.println("MQTT Connected"); mqtt.subscribe(mqttSub); //or mqtt.subscribe("topic"); /*with qos = 0*/ mqtt.publish("/news", "OLED display is online"); } void mqttDisconnected(void* response) { oled.clear(); Serial.println("MQTT Disconnected"); } void mqttData(void* response) { RESPONSE res(response); Serial.println("mqttTopic:"); String topic = res.popString(); Serial.println(topic); oled.println(topic); Serial.println("mqttMessage:"); mqttMessage = res.popString(); Serial.println(mqttMessage); } void mqttPublished(void* response) { } void setup() { // OLED initilize oled.begin(&Adafruit128x64, I2C_ADDRESS); oled.setFont(Adafruit5x7); oled.clear(); oled.println("OLED online"); // Hardware Serial (for ESP8266) and Serial Concole initialization Serial1.begin(19200); Serial.begin(19200); esp.enable(); delay(500); esp.reset(); delay(500); while(!esp.ready()); Serial.println("Setup MQTT client"); if(!mqtt.begin("TrainScheduleDisplay-LF", "admin", "Isb_C4OGD4c3", 120, 1)) { Serial.println("Failed to setup mqtt"); while(1); } Serial.println("Setup MQTT lwt"); mqtt.lwt("/lwt", "offline", 0, 0); //or mqtt.lwt("/lwt", "offline"); /*setup mqtt events */ mqtt.connectedCb.attach(&mqttConnected); mqtt.disconnectedCb.attach(&mqttDisconnected); mqtt.publishedCb.attach(&mqttPublished); mqtt.dataCb.attach(&mqttData); /*setup wifi*/ Serial.println("Setup WiFi"); esp.wifiCb.attach(&wifiCb); esp.wifiConnect(SSID, PASS); Serial.println("System Online"); } void loop() { esp.process(); if (millis() > (time + 10000)) { time = millis(); oled.clear(); oled.set2X(); oled.println(" IoL City "); oled.println(""); oled.set1X(); oled.println(" Train Network"); delay(2000); oled.clear(); oled.println(mqttMessage); if(wifiConnected) { // publish stuff here } } }

Download Source Code: Gist

Gallery