I had the role as product owner, so a big chunck of my time was spent moving around answering questions on how the app was supposed to work.

We where able to make great progress on a number of screens; creating an idea, creating a text note, localization, audio, images, lists of ideas with sorting, etc.

We discussed the limitations of Expo, but kept using it. I’ve had a short Twitter conversation with Brent Vatne that goes to the heart of it. In short; it seems that for payments we need to ‘eject’ eventually.

Lessons Learned & and revised future plans

While I see myself as a positive human being, I do like to focus on what goes wrong and should be fixed. Preferably in a structural manner. The biggest challenge now seems to be the first requirement:

Keep classic features, this includes being able to use the app without an account and while being offline.

Because we have over 6.000 existing users on Android we want to keep, this is a very hard requirement. In hindsight it seems we tried to to do too much at once. We should have developed Offline Only, before we tried Offline First.

Offline First, or Second?

A new way of thinking is starting to emerge in the web and mobile development scene, called Offline First.

Offlinefirst.org

We live in a disconnected & battery powered world, but our technology and best practices are a leftover from the always connected & steadily powered past. Offline capability is a key characteristic of modern Progressive Web Applications. Offline first thinking must learn from and further what we’ve seen work with Responsive and Mobile First thinking.

On closer inspection, they really are talking about what I call Offline Second. A web-app dealing with connectivity dropping, should at first work online.

My definition of Offline First, means you first build it offline. Without internet, without online accounts. Then when the whole app works, build on top of this code to add online capabilities second.

There is an architectural challenge in creating an offline first app; how do you synchronize the data? I’ll describe two different strategies in as much as a non-technical way as possible.

Strategy 1: Caching queries & using resolvers

One way to look at is, is to see the online database as the sole truth. You can ask questions (queries) to this database, and when the connection drops, you locally remember what the answers to those queries used to be. You use the cache storage to keep answers to the database around.

We use Apollo for our data management and have to be careful about how to use this cache. When you are offline, the last answer to a question is not always the correct answer.

To deal with this problem, you have to do some coding by hand for each query. You write resolvers that make sure the correct thing happens. The conversation then becomes like this:

After this conversation, the server could come back online and the local storage of answers to the queries would be used to sync the data to the server.

The nasty thing about this approach seems to be that you have to do create resolvers for every query and slowly start to build a local database in the form of resolvers. A local database full of entanglements that is hard to manage.

My biggest hope 🙏 is that I’m ignorant and someone comments that we just need to do X and Y to get a clean manageable architecture.

Strategy 2: Local database & communicate the difference to the server

While the first strategy leans on providing the online server with a list of changes to the local understanding of the database, the second strategy is to use an actual local database.

Of of the advantages is that it simplifies offline behavior. The app just uses a routine way to talk with the local database (SQL queries). Then a general piece of software, I’ll call a syncer, is placed between the local database and the online server. It keeps track of what is out of sync and makes updates if required. It wouldn’t matter what SQL queries you write to make changes to the local database, so now resolvers, the syncer would just look at the state of the database to see what is different.

This kind of solution reminds me a lot like how how version control like Git works for code management. Git is a tool that keps track of your code and make sure you have a copy on your computer that you can use offline, but is also able to sync with the code on a server.

You to start coding locally and then tack on Git aftwerwards. In the same way I’d like to first create an app using the local database, and then tack on a syncer later.

Some potential candidates

While I have a preference for the second strategy, we have to do more research. Perhaps there are more elegant solutions.

Some candidates:

Gun.eco

GUN is a small, easy, and fast data sync and storage system that runs everywhere JavaScript does. The aim of GUN is to let you focus on the data that needs to stored, loaded, and shared in your app without worrying about servers, network calls, databases, or tracking offline changes or concurrency conflicts.

I only learned about its existence during the hackaton. Skimming the texts, it’s hard to wrap my head around the impact of it being peer-to-peer.

GUN is fully decentralized (peer-to-peer or multi-master), meaning that changes are not controlled by a centralized server.

Firebase

Firebase gives you functionality like analytics, databases, messaging and crash reporting so you can move quickly and focus on your users.

Firebase offline data page

With offline persistence enabled, the Cloud Firestore client library automatically manages online and offline data access and synchronizes local data when the device is back online.

That ‘back online’ makes me feel it’s Offline Second. This quora page suggests it’s not really meant for Offline Only.

GraphQL resolvers with Apollo

Perhaps we missed something on how to make this work reliably, and we should just continue on the path we have taken.

CouchDB

This also looks promising, because the way they write about it, it actually is Offline First the way I defined it.

The Couch Replication Protocol lets your data flow seamlessly between server clusters to mobile phones and web browsers, enabling a compelling offline-first user-experience while maintaining high performance and strong reliability. CouchDB comes with a developer-friendly query language, and optionally MapReduce for simple, efficient, and comprehensive data retrieval.

It seems designed for data clusters, but then also mentions offline-first, so I’m somewhat confused on how it actually works. This article goes into detail how it could work with React Native, so i’ll have to look into that more.

Desired process

What could we have done better? It seems we may have tried to tackle to much at once. The ideal process keeps the momentum. At every step our team enjoys visible progress and we can keep learning and improving.

In hindsight, I should have aimed for the team to build an Offline Only app. We did a short evaluation, and this is our new roadmap:

Offline Only (simple Sqlite database)

Get the React Native app as soon as possible in the Apple store so we can start learning and improving. This means we start storing the ideas on the device only. While the classic Android app is also Offline Only, we don’t have an iOs app yet, so we can roll it out there first.

Get the React Native app as soon as possible in the Apple store so we can start learning and improving. This means we start storing the ideas on the device only. While the classic Android app is also Offline Only, we don’t have an iOs app yet, so we can roll it out there first. Back-up the full database

The current would be greatly helped if they could have a simple back-up feature for all their ideas. Now there is a quite tedious export/import feature. A decent back-up feature would give us the opportunity to learn about our users needs in this area and dealing with the server side, analytics, etc.

The current would be greatly helped if they could have a simple back-up feature for all their ideas. Now there is a quite tedious export/import feature. A decent back-up feature would give us the opportunity to learn about our users needs in this area and dealing with the server side, analytics, etc. Sync

Once we have the core working, we can build smart sync capabilities on top of that. Even before that, we can start experimenting with this right now in our one-text-app POC.

Once we have the core working, we can build smart sync capabilities on top of that. Even before that, we can start experimenting with this right now in our one-text-app POC. Teamwork

Then we move forward along the path discussed in earlier articles. Starting with teamwork.

There you have it

We had fun, made good progress, took two step forward and one step back. We calibrated our plans and now focus hard on first taking the app to the Apple Store as soon as possible.

If you have any suggestions for us on the Offline First challenge, please share your wisdom in the comments. I’ll keep this article up to date, incorporating new insights from you and others.

Update: The iOS version is live! ..and it’s no longer live, but you can read how it went.

More on our progress on our Twitter @ideaGrowr and on Medium over here: