I gave a talk about Android ORM’s at Droidcon UK yesterday a few weeks ago. Here are the slides:

https://docs.google.com/a/touchlab.co/presentation/d/1jZW-SKdlMmOZG2QrcnWNP7EPJj5bIunLXiHdUeVZq2U/edit?usp=sharing



Its a talk about ORM’s. On Android. Its a comparison of several tools for sqlite, as well as a bit about Realm.

There were some surprises. First off, I didn’t expect much of a turnout, because ORM’s seem to be out of favor lately (or maybe not?). Second there were multiple ORM authors in the room, including somebody from Realm in the front row. This I learned during Q&A ;)

I learned a bit about myself preparing the talk, and writing this follow up. First, I’m verbose. I knew that, but still. Second, I’m a perfectionist, so this is taking forever. Also not news. Third, I’d be a bad critic, because I get a little too critical once I get rolling. The tone went from a comparison to “let me tell you what you did wrong”, which is just kind of obnoxious. Entertaining, perhaps, but still. Also, right at the end, Realm put out some updates on numbers, so I’ve been revising and rethinking. Plus I have to run a business. So here we are.

I had a lot of content and deeper detail I didn’t get into during the talk. I also didn’t explain the methodology as deeply as I would have liked. This blog post fills in the gaps, and adds some context and follow up thoughts.

For background, I wrote the original adapter port of ORMLite for Android. Not ORMLite, just the Android adapter bit, plus some helper stuff. I’ve also done some explaining on the topic. After ORMLite, I just kind of moved on and hadn’t been paying attention to new tools, under the “if it ain’t broke” philosophy.

Android is entering what I’d call the “mature” phase. All frameworks go through similar cycles. When things are new, you can write basic tools and get lots of eyeballs, because there’s nothing else available. During the mature phase, there are many different library options available for everything. Think javascript. Many, many different libraries, for everything.

Its been a long time since I looked around at different options. I barely knew about other options, honestly. Then during Q&A after a talk in Berlin in 2014, somebody said GreenDAO was way faster. I dug a bit and found a blog post stating that ORMLite was 1200% slower than GreenDAO.

Benchmarks are funny. The benchmark author generally gets the result they want. In this case, the author added an eager foreign collection to the ORMLite example, but not the GreenDAO one. When I took that off, ORMLite was like 50%-100% slower, but that’s not 1200%.

That research kind of lit the slow burn that led us to now.

I use that same benchmark, with the issues fixed, to compare several libraries. Why do I use it? Well, mostly because I’m a wise ass. I think its funny that we’ll show that GreenDAO is relatively slow for a source gen framework, using the benchmark designed to make it look fast.

To be clear, I don’t know if the benchmark author worked on GreenDAO, or is just a fan. I could look into that, but I don’t really care enough. Regardless, they do like to say how fast they are. Notice they don’t include any source gen frameworks. Spoiler alert, they’d lose.

However, I think we all need to keep perspective. Raw performance is not the most important thing. Generally we (touchlab) use ORMlite to build an app without too much concern for performance, then test and improve as needed. Most apps adhere to the 80/20 rule in a few key ways. A few screens are used a lot, yet most are rarely seen. Also, a few features have serious performance demands, while most don’t. In practice, if you need to do any performance tweaking, its only in a few specific areas. You can always drop down to raw SQL if needed, and lets be honest. SQL isn’t THAT hard. Its a DSL for efficiently querying data. In fact, I’d say if you have serious problems managing raw SQL, you shouldn’t be using SQLite at all. But that’s me.

Having said that, if you’re building a library that will be used by many, and you can significantly improve performance without affecting complexity or flexibility, not doing that seems like kind of a waste.

So, my opinion is that you shouldn’t pick an ORM because its the fastest, but because of its features. However, I would encourage ORM authors to improve their performance where reasonable. If you’re not doing source gen, you should consider it. Or somebody should fork the framework and do that. You know. Make better.

Which leads to the anti-climactic part of the story. I started out seeing if I could write a custom Dao for ORMLite that was source-gen based, but after a bit of work, that seemed like a mess. ORMLite is all reflection based, and very much not Androidy. Its a generic database library. Instead, I forked ORMLite, trimmed much of the code, and made a “new” ORM called Squeaky with lots of ORMLite-isms. It shares a lot of code, and other code was ported. Lots of code was removed, along with some features I didn’t think really applied to mobile. Why anti-climactic? I’m not 100% sure we’re going to really “complete” the project. We use it internally, and the code is public, but there’s a lot of docs, tests, and feature reduction that still needs to be done. Does the world need yet another Android ORM? We’ve been working on some really interesting things lately, so its hard to justify the distraction, but we’ll see.

Features

All ORM tools need to be able to copy values to/from objects and tables. AKA marshaling (according to wiki, this may be an incorrect usage of the term, but whatever. You know what I mean). If your ORM can’t copy values from/to a database, its probably not very useful.

Some tools map relations between objects. This is the tricky bit. Its hard enough in a server ORM. You need to decide if you load lazy or eager, and use joins where they help. In Android, if you’re respecting thread boundaries, you shouldn’t be lazy loading. You’ll easily wind up with db calls in your main thread if not careful. Also, foreign collections suck. This is where a lot of people get the “ORM’s are bad” performance thing from. They’re not magic, so if your foreign collection has lots of data, polling it will be slow. Eager fetching data you don’t need is wasteful. The lack of bytecode instrumentation on Android also starts creating issues at this point. You can’t just shove in lazy loading mechanics. Long story.

I think its important to highlight the thread boundary issue here. If you’re strict about avoiding disk access in the main thread (and how strict you need to be is debatable), you’ll need to make sure all of your data is fully loaded before you pass it to the main thread. If you’re doing lazy loading, then you’ll need to try to grab the data in the background thread before passing it over the wall. That can mean simply accessing it in code, which is doable, but clumsy. Eager fetches, when defined in annotations, can be clumsy, because you don’t always need the data. Some frameworks always grab eager, and don’t have a depth killswitch, which is crazy. We’ll get to all that in a bit.

Most frameworks generate DDL. That’s CREATE, ALTER, DROP, etc. If they don’t, they’re pretty useless. Some hide SQL (mostly) and manage the db’s on their own, which implies generating DDL. I call this “Wizard of OZ”. Don’t look behind the curtain. I’m not a fan, so I’m hugely biased against anything doing that. Just FYI.

Most, or all, provide some form of query api. This is the free form jazz part of all frameworks. They’re oddly dissimilar, considering they all turn into the same thing at the end (SQL, except Realm, of course).

Lots of them provide other features, but we have to focus a bit. This is going to be long as it is.

Comparisons

The comparisons I’ve made are based on the following:

basic performance

model pollution

primary keys

foreign relations

integration difficulty

stability

threading



We’re only running numbers on basic table performance. Foreign relations is a whole different discussion.

Model pollution refers to how much the library intrudes on your models. Generally that takes the form of requiring the developer to extend base classes, hiding the SQLiteOpenHelper and SQLiteDatabase classes, and even requiring a custom Application class.

Only one of the libraries supports multiple primary keys. Some restrict you to numeric primaries, which can be problematic if you’re doing UUID’s, etc.

I’ll discuss how each framework supports foreign relations. Some are good, some are bad, some are worse.

Integration difficulty refers to how hard it is to get the library integrated and, to a lesser degree, how difficult it is to get your head around the library.

Some libraries are more stable than others.

Threading differs between libraries. Will chat about that quite a bit.

Implementations

We’ll be looking at the following:

ORMLite

GreenDao

DBFlow

Squidb

SugarORM

Cupboard

Squeaky

Realm

If you have another framework you think is totally bad ass, get in touch.

What am I looking for?

At a high level, I want a tool that sits on top of sqlite, provides some mechanical advantage, but doesn’t try to hide the underlying framework and/or hide sql entirely. It should help but not replace. There’s definitely a lot of my personal preference at work here. If you happen to love one of the frameworks that I don’t have kind words for, OK. I assume its working. Just not my brand of sauce. I try to be unbiased, but I’ve written one of these (two, kind of), so I’m totally biased. Its difficult to avoid.

Performance

This is a big topic when discussing ORM libraries, but a lot of the comparisons feel kind of bullshitty. For the most part, the author of a library will post a benchmark, and always, their library wins. That should not be much of a surprise. The problem is selective testing that will focus on the strengths of a library, or even avoiding libraries that outperform altogether. Also, if you see numbers and claims without code, you should be wary. It happens.

All of the libraries do basic data marshaling. That’s what we’re going to test. I’m not testing foreign references, at least not systematically. However, in general we’ll talk about their implementation and can draw reasonable conclusions from that. For sqlite, its pretty easy to figure out how a framework will compare just by looking at how it works. There are only so many ways to read/write data, and they either use reflection or source gen. The results track pretty closely with what one would expect. The outliers are SugarORM and, to some degree, SquiDB. I have no idea why the performance of both is significantly worse than would be expected, and since I would be unlikely to recommend either, I didn’t spend a whole lot of time on peeking under the hood.

I’ll throw in a special mention about Realm here. First off, its not sqlite. That means an A2A (apples to apples) comparison is not really possible. Does that mean its off the hook? Of course not! It just means we need to mention in what context the numbers might be inaccurate. It is more difficult to do an A2A comparison because the underlying tech is very different, but we’ll do our best (ish).

Anyway…

In the talk I point out that DBFlow, GreenDAO, and Realm all have posts declaring their performance superiority. Lets chat about that for a bit.

DBFlow

First off, no ORMLite? WTF?! I’m pretty sure it has way more installs than the rest, possibly combined. It wouldn’t “win”, but still. Perhaps you’re not a fan, but don’t hate! ORMLite spent most of its time not on github. That was relatively recent, so maybe less stars and forks. Whatever, anyway.

In my tests, DBFlow was the fastest on writes, but we’re talking within single digit percentage points of other options, and sometimes it would come in slightly slower. On reads, it came out slightly worse than some other options, but again, all of them were so close to “fastest” that comparisons are meaningless.

So, if DBFlow wants it, I declare them “fastest” on Sqlite. Victory!

GreenDAO

To be fair, they’ve been beating the source gen drum for a while. However, we need to face reality. If you’re not the fastest, and they are measurably slower, you should stop pushing that as a core feature.

They have a post about ORM performance in 2015. To do that comparison, they show you the first 2 ORM’s ever built for Android (AFAIK). This post should be renamed “Android ORM Performance in 2010″. That would be a more appropriate title. I know that’s harsh, but its true. Sorry. I’m not really a fan of DBFlow as a library, but if you put it in there, it would be significantly faster than GreenDAO.

Realm

Realm makes some serious claims to performance. I’ll explain this better later, but my general beef with Realm isn’t so much about the tech as about the marketing. I get a little bent out of shape when I read things that are arguably false or misleading. That makes me dislike the library before I even try it, and I have zero trust in the messaging coming out of the org with regards to potential issues. And lets be clear. A new database technology is a likely place for potential issues. I’ve lived through new db introductions on the server side, and have seen the nightmare support situations that can result. The Realm developers I’ve met are great, but I feel like the sales message kind of ruins things.

For example:

That’s the chart on their page about Android performance. I can tell you that’s wrong by looking at it. Why? As mentioned, I wrote the ORMLite-Android adapter code. I *know* ORMLite, really, really well. I’ve been playing with ORMLite and GreenDAO comparisons for a while now. Inserting basic table data is faster on GreenDAO. It just is. These numbers, where ORMLite is 2.5x faster than greenDAO, are not possible, unless they’re doing something really, really strange, and even then, I’m not sure this is possible. There’s no link to the benchmark, so I can’t test it (update below). If you’re going to claim performance numbers, you need to link to your benchmark. I made a similar chart using my numbers.

Its not pretty. Besides my lack of gradients, the numbers aren’t as exciting. Are these numbers true? Its a benchmark, and benchmarks are bullshit. I use ‘copyToRealm’ due to the construction of the test. Is that slow? No idea. Sqlite uses a prepared statement instead of ContentValue, and I’m sure Realm’s benchmark used ContentValue (again, update below with new benchmark). I’m trying to insert records on each platform in the fastest way I know how to, and with Sqlite/Android, using a prepared statement is the fast way to do it. If you want something even less pretty, here’s the full stable of frameworks tested:

Still, though, plenty of bullshit involved, because its a benchmark that I wrote. If you have time, please fix the benchmark and do a pull request. I will absolutely update my post (assuming I don’t get hammered with pull requests, but lets be honest, I think the only people still reading are me and the Realm folks).

Again, the Realm performance page has been updated with a link to more discussions about performance, but you still can’t verify the main page numbers.

I ran this on a OnePlus Two. I’m assuming reflection has improved over time, which is giving ORMLite better numbers than might have been observed in the past.

And, yes, spoiler alert. SugarORM is horrible, and not just its insert performance.

ORM Hate

I made a point in the talk to not get into this too much. I hear about it a lot. Most of that feels like repeated statements, not experiential opinions. Primarily they fall into performance and the object/relational mismatch.

I worked for a guy who did a lot of work on Hibernate. “Hibernate sucks” is repeated a lot, but he convinced me that in almost all cases, the user was doing something bad and not understanding it. Hibernate is highly tuned, but ultimately it deals with SQL. It gives you opportunities for significant mechanical advantage, but lets you potentially do damaging things. Not going to try to convince you that ORM’s can be OK. If you don’t think so, you won’t agree, and high five. I’m sure your app will be fine.

The mismatch is a trickier issue. Hard to argue with people about this. I think its mostly one of expectations. Expecting an ORM to map any object model to relational tables, without affecting the model, is expecting a lot. They’re different things with a similar vibe. At some point, you have to bend one to the other. You can manually adapt your data to SQL, or you can accept a compromise on your OOP models. We’ve accepted the latter. You don’t have to, and I’m sure there will be no convincing you to do so (and no reason to try).

The ORM is a tool that may help you build something. Its not perfect, but also not required. Lisa Wray of Genius (ex Google and NYT) pointed out that there’s a bit of an obsession with best and perfect when discussing tools. I’ve noticed that myself. If you work at a large company with many engineers, you might have time for this. She’s the only developer on their app now (they’re hiring). In those kinds of situations, you have to make compromises. Big tech is disproportionally represented in tech discussions, and they disproportionally have resources to try “perfect”. I’m not even going to try to support that statement with evidence. Just throwing it out there ;)

The Frameworks

None of these will be treated fairly. I don’t really have time to go through multiple scenarios and code examples. I only got a surface understanding of each, if I didn’t know it deeply already. If anybody reads this, and its a damn long post already, I’ll try to write a follow up post about specific frameworks if there’s interesting feedback. Again, if the benchmark is wrong, please submit pull requests.

ORMLite

I worked on this. That makes me hugely biased. Trying to do my best.

ORMLite is a generic DB framework. Its not Android specific. That means there are features you don’t really need, and potential optimizations missed. That also means extra methods for the dex count.

Its Mature. Lots of apps use it. Are bugs occasionally found? Yeah, but that’s code. Its damn stable. Lots of support on Stack Overflow.

Its reflection based. That makes it slow-ish. I tested on Lollipop, which I suspect has improved reflection quite a bit. Also, Gray Watson has optimized the hell out of something that uses generic interfaces and has to deal with many types of databases. Besides raw performance, though, I’m a little concerned about transient object creation. Values are moved through Object state, even if they might be represented as primitives. So, if you have an int field, it’ll be an Integer in the transform. Not sure if that’s a huge issue.

Single primary key (I’ll save time. Only DBFlow can have multiple). String or integral keys.

Low/medium model intrusion. You don’t need to extend model objects. You do need to extend the SQLiteOpenHelper provided. If you consider annotations to be “intrusion”, then yeah, lots of that.

There’s a whole set of apptools Context base classes. The idea was to extend them, and they’d manage your db lifecycle and close when you’re done. They would be intrusive, but you shouldn’t use them. I wrote them. They’re bad. Don’t use them. It was one of the first things I did on Android. You don’t actually need to close your SQLiteOpenHelper, so they don’t serve a purpose.

Foreign references are OK. You can do lazy or eager fetch. If you have eager marked, it’ll go down 2 levels by default, then stop. That’s blunt, but prevents object graph nightmares. If you have a singular foreign, there’s no programatic way to determine that it hasn’t been refreshed. It might be a stub with just the ID applied, but no way to know for sure if it was actually filled or not. That’s not ideal, but simple.

Foreign references do not do an optimized query with a join. They trigger another db call. In theory you can grab singular foreign references with a single query and join the tables, but very few of these frameworks do that.

I don’t know if ORMLite can handle views. It *might* treat them like another table. Probably would. To be attempted.

GreenDAO

This one is funky. First off, its complete model intrusion. You don’t actually write model classes. You write builder code, run that somewhere undefined in your build process, and model classes are generated for you.

This is definitely a personal preference kind of thing. I *hate* that model. Some people *love* it.

Its fast-ish, but as discussed, definitely not the fastest of the source-gen crowd.

Setup is kind of weird due to the “out of bounds” definition step. Where does it go? Its not really that big of a deal. Just write a main() method and run it occasionally, but still. Kind of weird.

Primary keys can only be long/Long. No String. That will turn off some.

Its the only library that supports some form of optimized join for eager fetch on foreign relations. However, its a specific method call, and (I think) limited to specific cases. Do not quote me on that, though. Investigate if that’s your thing.

If you don’t like ORMs, but don’t want to write your own SQL, this is an interesting option. Use it as a transform between your perfect OOP design and the relational tables. That’s about the best use case I can think of outside of people who simply prefer to do things this way. Again, though, huge bias.

DBFlow

As discussed, its fast. No BS or reservations here. On basic table ops, its really fast.

Medium+ model intrusion. This has to do with lots of “Wizard of OZ” situations. You can’t get at the open helper, db’s are defined in a proprietary way, migrations are totally different. That kind of thing. Hiding the underlying framework is unnecessary to me. Swapping syntax without demonstrably improving the situation doesn’t add obvious value.

The real WOOZ thing that bothers me is with transactions. If you read the docs, they suggest you use their transaction manager, which is a transaction and thread pool manager wrapped up in a single thing. There’s no simple “run in transaction” example. Also, I think that implies simple queries should be on the main thread? Hmm. That’ll be controversial. The thread manager has things like priority, which seems weird. Why? Also, most sophisticated apps have their own thread management. Mixing those is really bad practice. DBFlow *needs* to first provide basic transaction helpers, then OPTIONALLY provide a whole app/thread management library. They’re different things.

Multiple primary keys. That’s different. Some people really prefer that for relational databases, and I don’t have a good reason to disagree (besides that I rarely do it, even on the server).

Foreign relations are poor. They are verbose, and don’t have a killswitch. That means you can select a lot of data you don’t need, and it triggers a recursive call to the db. I made an admittedly shitty test where I created a self-reference on Message, then set the parent message on each to the one before. That meant a 20k deep parent/child relationship. If you just try to get the last one, it’ll try loading recursively up the chain. Imagine what would happen if you tried to load all.

They mention caching. Its not necessarily bad, but in general, caching is the fastest and most efficient way to get stale data into your app. Ha ha, but seriously, be careful with caching.

There’s a content provider generator. I’m seriously against using a content provider by default, unless you really need to share data outside of your app. Lots of people disagree, though, so maybe this is your thing.

SugarORM

Um, sorry. Apologies in advance. Hated it. Here’s a puppy.

Its simple, which is nice. The docs start very simple, and I assumed they’d reveal the complex bits later, but there aren’t really any complex bits. That’s problematic, as the realities of the problem require some config and safeguards.

Speed, slow. See results.

High model pollution. You need to extend model classes. You also need to define a custom Application class, which will manage your Database. Although some say this is bad practice, I’m not against custom Application classes per say. Just frameworks that require you to use theirs. Config is in AndroidManifest.xml, using meta tags, which is weird, frankly. It also means only 1 DB is possible. Migrations are custom. Open helper, hidden.

Transactions are mostly ignored in docs, which is insane. In the source, there’s a transaction helper you can use, but it catches Throwable, and logs that to debug, then moves on. If that’s an indication of the general framework error philosophy, that’s nightmare town.

Id columns are fixed, but they’re called ‘id’. Why not ‘_id’? Parts of Android expect that. If you’re going to name the column arbitrarily, I can’t figure out why it wouldn’t be ‘_id’. Right? Cupboard also fixes the id column name, but its ‘_id’. Its just the obvious choice.

Foreign relations have no kill switch, so its easy to select large parts of your object model if its not super simple. Just throwing that out there. Maybe that’s less of an issue than I think it is, but can get you into trouble.

I like that the package is ‘com.orm’. Shows confidence!

If you like sugar, OK, but be careful. I think its too simple, and its really does a lot of Wizard of OZ, for no obvious benefit. Fixing what isn’t broken is exhausting work.

SquiDB

This is a strange framework. You write “spec” classes, and your fields define the entity fields. Then you run apt, which generates the classes you actually use. In that sense, it breaks the 4th wall, as you actually need to directly include generated code. You don’t use the classes you write, just the generated classes. The classes you write, then, I guess, are dead code? Its complicated, but doesn’t support foreign relations, and compared to other frameworks in its class/generation, its slow. Not really sure who its for. I am probably not seeing its killer feature, so if you know what it is, hit me up.

Squeaky

Squeaky is an annotation processing port of ORMLite. It started as an attempt to slighly modify ORMLite to marshal data with generated source. While doing that, it was pretty obvious that tons of code could be yanked out of the framework and an Android specific version could be pretty sweet (if you think ORMLite could ever be sweet). I also hadn’t done a major coding exercise in a while, so I tucked in.

The name. My old apartment had mice periodically. I’d occasionally see one while coding in the wee hours. The name starts with ‘SQ’. That’s it. I need better library names.

Its source gen, but maintains ORMLite-isms. If you’re an ORMLite user, it would be fairly drop-in as a replacement.

Its fast. DBFlow and Squeaky are neck and neck. When I looked at DBFlow generated code, they’re largely the same basic structure. I did this after it was implemented, so great minds think alike? That, or its an obvious design problem. I will admit. For inserts, there’s some small edge I can’t figure out on DBFlow. Granted, I was trying on the flight to London, and its minimal, but its there. Damn you DBFlow! And congrats, I guess ;)

It supports final fields, but only on the top level object. Foreign relations do not support this due to the loading structure (currently). To have final fields on a data object, make sure they’re in the constructor, with the same name and type. It doesn’t have to be public, but can’t be private.

It supports views, but also a new feature, which is an ad-hoc query. You put the from SQL query in the annotation, and you can treat that query like an inner from clause. Useful for join queries and such, without defining a View.

The one distinguishing feature is the foreign auto-refresh map. The idea is that, if you’re respecting thread boundaries for data loading on Android, you want to grab all of the object graph you’ll need before going back to the main thread, but not more than you’ll need. You can pass a simple map to the query, which is just a simple string of field names (can be nested), and it’ll follow that when refreshing data (it overrides eager fetch settings on the annotations, if the map is provided. By default the query generates a map internally that respects the eager fetch annotation settings).

When building apps, I tend to develop first without much thought to speed, then optimize later. The auto-refresh map gives you a fairly straightforward way of doing that. Start off simple, then be explicit about what data you do/don’t load.

The code is open source, but as discussed, not sure if we’re pushing it too hard. See how it goes.

To add:

* Optimized foreign joins

* SQLCipher support

* Better foreign collection handling

* “Live” mode, which would basically be main thread data access, but simpler object graph navigation. Not sure this will ever happen, but it makes design simpler

Realm

I’ve been pretty harsh on Realm, but I want to be clear. I think the tech is interesting, and I personally really like the developers I’ve interacted with. I just hate the marketing, and as the marketing sort of dumps on things I’ve personally worked on, I feel compelled to respond. Well, mostly I don’t care, but I’ve been reading and re-reading the same posts for weeks, so I’ll just kind of vent for a bit.

In defense of Realm, if we assume its stable, and if we can assume main thread data access isn’t actually an issue, its got a lot of nice things going for it. Because foreign references are lazy loaded, coding is simple and doesn’t need to worry about object graph nightmares. If you don’t read all fields on an object, its proportionally faster to load, so less wasted access time. Its probably a lot better with memory, although the data lives in the “dark matter” realm of jni, so that’s hard to measure. I’m sure the fact that data isn’t stale between threads is somehow an advantage, although you have some long-standing habits to unlearn with that stuff.

However. I haven’t talked to a “senior” dev who uses Realm, or is interested, and that’s kind of an issue. The marketing problems I mentioned above won’t be noticed by less experienced developers, but seasoned Android people notice them almost immediately. And we talk. There’s a general disconnect between marketing people and developers. We’re doing professional level work, and making decisions that will affect both the product we’re working on and our reputations, not to mention our free time. I’m reading your docs like a doctor would read a medical study. I’d pay attention to how Kotlin is being marketed to developers.

An example:

“First of all it’s always available, even when the user is offline. Secondly, it’s super fast, and because it’s local, more private — you’re not sharing your data with a database sitting on someone’s external servers. It also saves money because as Stigsen points out, developers typically pay for those external database calls.”

http://techcrunch.com/2015/03/24/realm-can-expand-its-reach-with-20m-investment/



That kind of sums up my issues. Those aren’t false statements, but they’re also not distinguishing features. All local db’s do the same thing. If Kotlin said, “you can create methods and objects to better organize your code”, that would be true, but also a pointless statement. See what I mean? It feels targeted at newer developers. I don’t know. Whatever. Its weird.

Yes, its a techcrunch article, so maybe the author was reading into things a bit, but if you read this, not so sure:



https://realm.io/news/introducing-realm/



I won’t give you a full annotated read, but if you scroll to “What does it mean for you, the developer?”, point 1 is used to set up the idea that developers don’t actually use local db’s, because they’re too hard, which then paves the way for 3 & 4, which are only meaningful if 1 is true. That’s called a logical fallacy. Probably straw man?

I would recommend to the Realm PR giving a more balanced description of features and benefits when presenting the framework. For example, yes, you can share the db file between iOS and Android. However, I’ve never needed that, and I run an Android consulting company, which means I’ve been in a lot of multi-platform situations. Also, AFAIK, core data runs on SQLite (mostly), so can’t you share that anyway? I bet you could, but again, I’ve never needed to do that. Its a non-feature.

The original benchmark has been updated as of 11/6, if you look at the blog post here:

https://realm.io/news/realm-for-android/



Below the huge chart showing Realm much faster on inserts, there’s a link to updated benchmarks. Its a good step, but the huge chart showing questionable numbers is still huge and front and center. You know what I mean? It doesn’t feel entirely honest. Granted, I’m reading and re-reading the same posts over and over, so I’m noticing it far more than most, but still. Change the chart, maybe.

This is a screenshot from the link. Notice on the far left. That’s “batch write”. Looking at the code, that’s 1000 rows, and sqlite and Realm are basically the same result. That’s a lot different than the other big chart, yes? The big numbers on the right in this chart are with sum and count, which are aggregate operations, and yes, Realm definitely does that faster. I think I mention that above, but honestly, this post is getting away from me. I said that in the talk for sure.

I hadn’t planned on reviewing another benchmark, so I’m just kind of scratching the surface. I might dig in and try to resolve discrepancies, but I don’t really care. The point is that these numbers, my numbers, and the numbers on the original post are WAY off, at least for inserts. That creates kind of a trust issue, and if you’re going to base your app on a technology, trust is fairly important.

If you want to appeal to expert level folks, my thoughts. First, convince them that main thread data loading is OK. For the old timers, reading data on the main thread is heresy. It *might* be OK, but you need to prove that. That whole topic is glossed over, at least in info I’ve seen. Not sure how you would convince people of this, but you just kind of have to. On the crappy devices. Not the brand new stuff.

Yes, AsyncQuery was added, but even the docs about it say it should only be used in exceptional cases (and data grabs aren’t async, because that’s not how Realm works, yes?).

Second, I’ve lived through new db “revolutions” before. Its a risk, and the first wave tend get hurt. That’s just how it goes. You’re trying to replace something that works just fine for most, contrary to the claim of “2–3 months of every 6-month project” blog post linked above. Why should they switch? My main concern is really that, to be honest. Not sure what you can do about that, but its a risk.

What do you NEED to do? Probably nothing. Not too many people really hammering the issue. If you had left ORMLite off the chart, I probably wouldn’t be writing this.

The interesting thing, until recently I didn’t realize that Realm can store data encrypted on disk. Considering the general state of security, especially on Android, I can’t imagine why that isn’t pushed much harder. We’re working on ResearchStack, and all the data will probably need to be encrypted. None of the ORM’s currently support SQLCipher. So, we might be using Realm. The irony.

Other Thoughts

I just kind of realized we didn’t talk about more complex data types. Dates being the only thing I think is really useful. Also, I tested Cupboard, but didn’t get enough time to update the talk on my thoughts. I’d like to play with it more, but if you don’t want foreign relations, are OK with a fixed id column, and prefer a lightweight solution without the bells and whistles, its a good option.

Conclusion

Not sure. None of the options is really the obvious choice. AFAIK, they’ll all work, and if your app isn’t doing anything crazy, it probably doesn’t matter much which you choose. We’re using ORMLite still, as well as Squeaky, but that should be obvious. Picking between ORMLite, greenDAO, and DBFlow is probably more personal preference than anything else, and they’ve all got their fans. If you enjoy pre-release code, I’ll update with the Squeaky repo details. If you just want basic cursor handling, look at Cupboard.

This also kind of turned into a big talk about Realm, which wasn’t the goal. Outside of the medical study framework, I don’t think we’d be likely to use it in a production app, but we have a lot more sqlite experience in house. For an encrypted db, its a different discussion. I have no sense of the stability of SQLCipher. As scary as the idea of a corrupt unencrypted database is, if something goes wrong with an encrypted one, you can pretty much just go home. Encrypted DB’s is an interesting topic, so if there’s another DB post, it’ll probably be about that. TBD.

Update

OK, doing more Squeaky work. Added SQLCipher support. The syntax is a little rough, but it appears to work. I updated the benchmark to try Squeaky and Realm encrypted. SQLCipher is slower than I expected on reads. Almost as slow as writes. Short, barely tested results, put Realm slower on writes, faster on reads. Very, very preliminary, though. Its basically at the “it compiles!” stage.