Lessons

I’ll talk about some of the lessons I learnt, and provide some reasoning or backstory to them.

Don’t trust anything. Don’t make any assumptions.

One assumption that was made was while building a acceleration and deceleration profile for the train — that the train would stop where we expected to. Calibration was done by running the train at full speed around the tracks, and when it hits a sensor, tell it to stop.

That assumption turned out false: at max speed, the train varies its stopping distance quite a bit, up to ±3 cm. Not only that, the train accelerates and decelerates non-linearly!

Real systems are often imprecise. For example, look at the stopping distance of a train. In our experiments, it never stops in the same place — for a variety of reasons: such as buffering, interrupt handling, incorrect system timing.

This meant that over the course of several start and stop move commands, there can accumulate quite a difference between where you think the train is, and where it actually is.

Consequently, our model couldn’t be trusted. We’d assume it was safe to perform a certain operation, and it of course, wouldn’t be. And that causes accidents like multi-track drifting to occur.

Multi-track drifting. Happens when a track switch flips direction while the train has yet to clear both axles.

Multi-track drifting is when you thought it was safe to flip a switch, and then it wasn’t. The poor train comes to a sudden stop because the front axles and rear axles are going to different directions.

When your model of the world and the real world itself are different, you get problems. We couldn’t trust our model. We’d assume it was safe to perform a certain operation, thinking we knew where the train was, and it wasn’t safe.

This explains the comic that is pasted on the walls beside the train set. Now I get it!

A printed out version of this comic was taped to the wall beside the train tracks. The original multi-track drifting.

If this was a real train, this would result it some serious accidents. A lesson we learnt after trying to figure out how this happened.

Assumptions are going to be wrong! So it’s important to build in error correction and recovery.

Often assumptions are wrong, or become wrong during the course of real life usage. Real systems don’t always run in your ideal situations. You can’t always think of all the edge-cases. Errors are going to happen. Your best chance is to try to realize the error as early as possible and correct for it.

Instead of eagerly switching tracks for the next train, we switched tracks only if the train had ownership of the track.

Map of the train switches and segments. We had two tracks, A and B, and they have different configurations. We hard coded the map to build the train track reservation system.

To solve this, we ended up building in margin of error around where we think the train is, assigned the train to have ownership to tracks, and made sure when trains approach tracks that they didn’t own, it would stop.

You are as only good as your tools.

Building the path/route finding portion of the project, we were manually reading a map and trying to reason about what the shortest ways were. This took a long time, and felt complicated.

Route a train from point A to point B was not difficult, but factoring in what paths are currently, or will be, blocked by other trains made it harder. We often stared at the layout map for a long time, trying to figure out whether this was the quickest path through this or not.